Discuss the differences between stochastic gradient descent (SGD) and mini-batch gradient descent.
Instruction: Compare and contrast these two variations of gradient descent, including their advantages and disadvantages.
Context: This question aims to assess the candidate's understanding of nuanced optimization algorithms and their practical implications in training machine learning models.
Updated
Official answer available
Read the opening below, then unlock the full answer and practical guidance.
Using the strict one-example terminology, stochastic gradient descent updates from one sampled example at a time, which can make updates noisy...