Derivatives can be approximated from nearby function values when an analytic derivative is unavailable or when the function is known only at sampled points.

For a step $h$, the forward difference is

$$f'(x)\approx\frac{f(x+h)-f(x)}{h},$$

while the central difference is

$$f'(x)\approx\frac{f(x+h)-f(x-h)}{2h}.$$

Taylor expansion shows that the leading truncation error of the forward difference scales proportionally to $h$, while the central-difference error scales proportionally to $h^2$ for a sufficiently smooth function.

Smaller $h$ reduces truncation error only up to a point. Subtracting nearly equal floating-point values can amplify roundoff, so an excessively small step can make the approximation worse.

Finite differences illustrate a fundamental numerical trade-off between discretization error and finite-precision error.