Learning from finite data requires assumptions about which patterns should generalize beyond the observed examples. Those assumptions are a model's inductive bias.

A predictor may, for example, prefer relationships that are approximately linear, assume nearby inputs should have similar outputs, favor simple threshold rules, or prefer smoother functions over rapidly varying ones. Each choice makes some patterns easier to learn and others harder.

There is no universally best bias. Informally, no-free-lunch results say that when averaged over all possible prediction problems, an algorithm that performs better on some problems must perform worse on others. Success comes from matching assumptions to structure in the problem.

This explains why asking “which algorithm is best?” without specifying the data-generating situation is incomplete. Model selection should instead ask questions such as:

• Is locality meaningful?
• Are approximately linear effects plausible?
• Are interactions and thresholds important?
• How much data are available relative to model flexibility?

Inductive bias is not a flaw to eliminate. Without some bias, observed data cannot determine how a predictor should behave on unseen inputs. Good machine learning makes those assumptions appropriate, testable and visible.