A statistic is calculated from a sample, so a different random sample generally produces a different value. The sampling distribution describes that sample-to-sample variation.

Suppose a statistic is written

$$T=T(X_1,\ldots,X_n).$$

If the same sampling process were repeated many times, each sample would produce a new value of $T$. The probability distribution of those possible values is the sampling distribution of $T$.

Data distribution versus sampling distribution

The distribution of individual observations describes variation among the measured units.

The sampling distribution of a statistic describes variation among samples or estimates.

For example, individual heights in a population may vary widely, while sample means from groups of $100$ people vary much less.

Center and bias

An estimator is unbiased for a parameter $\theta$ when its sampling distribution is centered at the true parameter:

$$E[T]=\theta.$$

Bias is therefore a property of the repeated-sampling behavior of the procedure, not merely whether one particular estimate happens to be close to the truth.

Standard deviation versus standard error

A standard deviation describes spread within a probability distribution. For observed data, the sample standard deviation $s$ describes how far individual observations tend to vary around their sample mean.

A standard error is the standard deviation of a statistic's sampling distribution. It describes how much an estimate would vary if the sampling procedure were repeated.

These answer different questions:

standard deviation -> how variable are individual observations?
standard error     -> how variable is this estimator across samples?

Standard error of the sample mean

If $X_1,\ldots,X_n$ are independent observations from a population with standard deviation $\sigma$, then

$$\operatorname{SE}(\bar X)=\frac{\sigma}{\sqrt n}.$$

When $\sigma$ is unknown, it is commonly estimated using the sample standard deviation:

$$\widehat{\operatorname{SE}}(\bar X)=\frac{s}{\sqrt n}.$$

The $1/\sqrt n$ relationship shows why averaging more independent observations usually makes the sample mean more stable.

This formula is specific to the sample mean under the stated assumptions. Other statistics have different sampling distributions and therefore different standard-error formulas or estimation methods.

Standard error is not measurement spread

A small standard error does not imply that individual observations are tightly clustered. A population can have substantial individual variability while a mean estimated from a large sample has small sampling uncertainty.

Conversely, standard error does not describe systematic bias: an estimator can be very precise across repeated samples but consistently centered on the wrong value.

Sampling distributions and standard errors provide the probability model that connects observed sample statistics to uncertainty about population parameters. They form the basis for later ideas such as confidence intervals and hypothesis tests.