A discrete-time Markov chain is a sequence of random states $X_0,X_1,\ldots$ whose next-state distribution depends on the present state but not on the earlier history once the present is known: $$P(X_{n+1}=j\mid X_n=i,X_{n-1},\ldots,X_0) =P(X_{n+1}=j\mid X_n=i).$$

For a finite state space, transition probabilities form a matrix $P$ with $$P_{ij}=P(X_{n+1}=j\mid X_n=i),$$ where every row sums to one. If the current state distribution is the row vector $\pi_n$, then $$\pi_{n+1}=\pi_nP,$$ and after $k$ steps $$\pi_{n+k}=\pi_nP^k.$$

Consider a two-state weather model with states sunny $S$ and rainy $R$: $$P=\begin{pmatrix}0.8&0.2\0.4&0.6\end{pmatrix}.$$ If today is certainly sunny, $\pi_0=(1,0)$, then tomorrow's distribution is $$\pi_1=(1,0)P=(0.8,0.2).$$ A stationary distribution $\pi$ satisfies $$\pi=\pi P.$$ For this chain, solving together with $\pi_S+\pi_R=1$ gives $$\pi=(2/3,1/3).$$

Under suitable irreducibility and aperiodicity conditions, the distribution of the chain approaches a unique stationary distribution regardless of the initial state. These conditions matter: a chain split into disconnected classes or forced to alternate periodically need not converge in this way.

Markov chains model systems whose evolution is stochastic but state-based. Their transition structure, stationary distributions and convergence properties underlie queueing models, stochastic algorithms, statistical physics and Markov-chain Monte Carlo.