Gradient Methods

When to Use

Use this skill when working on gradient-methods problems in optimization.

Decision Tree

1. Basic Gradient Descent
   • Update: x_{k+1} = x_k - alpha * grad f(x_k)
   • Step size alpha: fixed, diminishing, or line search
   • Convergence: O(1/k) for convex, linear for strongly convex