When a probabilistic model contains unobserved, or latent, variables, direct maximum-likelihood optimization can be difficult because the likelihood sums over unknown assignments. Expectation-maximization (EM) alternates between estimating those assignments and updating the parameters.

Let observed data be $x$, latent variables be $z$, and parameters be $\theta$.

The E-step computes the posterior distribution over latent variables using the current parameters:

$$q(z)=p(z\mid x,\theta^{old}).$$

The M-step chooses new parameters that maximize

$$Q(\theta)=\mathbb E_{q(z)}[\log p(x,z\mid\theta)],$$

the expected log-likelihood we would use if the latent variables were observed as well as the data.

For a Gaussian mixture model, the E-step computes component responsibilities for every point. The M-step recomputes mixture weights, means and covariances using those responsibilities as fractional membership weights.

Each EM iteration does not decrease the observed-data likelihood, but EM can converge to a local rather than global optimum. Initialization therefore matters.

The broader idea is reusable: alternate between inferring hidden structure under the current model and fitting parameters as though that inferred structure were available probabilistically.