Compute gradient-descent updates and diagnose an unstable step size.
Instruction: For L(w) = (w − 3)², start at w = 0. Compute two full-gradient descent updates with learning rate 0.1 and the loss after each. Then explain what happens with learning rate 1 and with a rate greater than 1.
Context: Tests the update equation, exact loss calculations and the distinction between a descent direction and a safe step size.
Updated
Official answer available
Read the opening below, then unlock the full answer and practical guidance.
The gradient is 2(w − 3), and the update is w_next = w − eta × 2(w − 3).
With eta = 0.1, the first step is 0 − 0.1 × (−6) = 0.6, giving loss (0.6 − 3)² = 5.76...