$k$-nearest neighbors (k-NN) predicts from the training examples closest to a new input instead of fitting an explicit global formula.

Given a distance $d(x,x_i)$, find the $k$ training examples with smallest distance to query $x$.

• For classification, predict by majority vote or a distance-weighted vote.
• For regression, average their target values.

Suppose the five nearest labeled examples contain four class A points and one class B point. A 5-NN classifier predicts class A.

The choice of $k$ controls smoothness. With $k=1$, the model can follow local details closely and have high variance. Larger $k$ averages over a wider neighborhood, increasing bias but reducing sensitivity to individual examples.

Distance makes feature representation crucial. If one coordinate ranges from 0 to 100,000 and another from 0 to 1, Euclidean distance will be dominated by the first unless scales are handled deliberately.

Prediction can also become expensive for large datasets because the method stores training examples and searches among them at inference time. k-NN is therefore conceptually simple but exposes important general lessons about locality, scaling and dimensionality.