To minimize a differentiable objective $f(x)$, gradient descent repeatedly moves opposite the gradient:

$$x_{k+1}=x_k-\alpha_k\nabla f(x_k),$$

where $\alpha_k>0$ is the step size.

The gradient points in the direction of steepest local increase under Euclidean distance, so $-\nabla f$ is the steepest local descent direction.

Gradient descent is iterative: each step uses local slope information to construct a better candidate rather than solving the whole optimization problem at once.

A useful step size can decrease the objective, while a step that is too large can overshoot or diverge and one that is too small can make progress extremely slow.

Because the method uses local information, its global behavior depends on the geometry of the objective. Different initial points can approach different stationary points when the objective has several valleys or flat regions.

Gradient descent defines the direction rule; line searches, fixed schedules and stochastic gradients are additional choices layered on top of it.