A generative model represents a probability distribution over possible data rather than only predicting a label or numerical target for a given input.

For images, the model conceptually assigns probability density across a very high-dimensional space whose coordinates are pixel values.

Modeling data distributions

Real observations occupy structured regions of that space. A useful generative model should assign high probability to data resembling the training distribution and low probability to implausible configurations.

Sampling

Once a distribution has been learned, generation means producing a new sample from the model.

Different model families accomplish this in different ways: some transform a simple random variable into data, some generate components sequentially, and diffusion models iteratively transform noise toward the data distribution.

Conditional generation

A model can also represent a conditional distribution such as

$$p(x\mid c),$$

where $c$ may be a class, text prompt or other context. Generation then samples data consistent with that condition.

Likelihood and implicit models

Some generative models provide a tractable probability density or likelihood, while others define a sampling process whose exact density is difficult to evaluate.

The central distinction is that a generative model aims to reproduce a distribution of possible observations, not only one deterministic prediction.