Standard supervised learning usually assumes that training and future examples come from the same relevant distribution. Distribution shift occurs when that assumption fails.

Several forms are useful to distinguish:

• covariate shift: the distribution of inputs $P(X)$ changes;
• label shift: class prevalence $P(Y)$ changes;
• concept shift: the relationship $P(Y\mid X)$ itself changes.

A fraud model trained before a new payment technology is introduced may encounter new input patterns. A medical model trained in one hospital may see different patient demographics elsewhere. Even excellent random-split test performance does not guarantee robustness to these changes.

Evaluation should therefore mimic the intended deployment boundary whenever possible: later time periods, different sites, new devices or distinct user groups may be more informative than a purely random split.

Monitoring after deployment is also part of the learning problem. Changes in feature distributions or predictive performance can indicate that retraining or model redesign is needed.

Generalization is always relative to a distributional assumption; “unseen data” is not enough if the unseen data are drawn from the same narrow conditions as training.