A model can fail because it is too inflexible for the structure in the data or because it adapts too closely to accidental details of the training sample.

Underfitting occurs when the model family or training procedure cannot capture enough of the real pattern. Training error and held-out error are both high.

Overfitting occurs when training performance becomes much better than performance on unseen data. The model has learned sample-specific variation that does not generalize.

A useful conceptual decomposition is the bias-variance tradeoff. Restrictive models tend to have higher bias: their predictions are systematically limited by simplifying assumptions. Very flexible models can have higher variance: changing the training sample can change the fitted predictor substantially.

For example, fitting a straight line to a strongly curved relationship may underfit. Fitting a very high-degree polynomial through a small noisy sample may pass almost exactly through every training point yet oscillate badly between them.

More complexity is therefore not automatically better. More data, better features, penalties that discourage unnecessary complexity, and careful model selection can all change this tradeoff, but their success must be judged on held-out data rather than training fit alone.