When two random variables are considered together, we may want to know whether values above the mean of one tend to occur with values above or below the mean of the other. Covariance measures this tendency.

For random variables $X$ and $Y$,

$$\operatorname{Cov}(X,Y) =E[(X-E[X])(Y-E[Y])].$$

An equivalent formula is

$$\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y].$$

Interpreting the sign

If large values of $X$ tend to occur with large values of $Y$, and small with small, the product of their deviations is often positive and covariance tends to be positive.

If large values of one tend to occur with small values of the other, covariance tends to be negative.

A covariance near zero indicates little linear co-variation, but it does not imply that the variables are independent.

Independence

If $X$ and $Y$ are independent and the required expectations exist, then

$$E[XY]=E[X]E[Y],$$

so

$$\operatorname{Cov}(X,Y)=0.$$

The converse is false in general.

Units and scale

Covariance has units equal to the product of the units of $X$ and $Y$. Rescaling either variable changes the numerical covariance:

$$\operatorname{Cov}(aX,bY)=ab\operatorname{Cov}(X,Y).$$

Because its magnitude depends on scale, covariance is most useful as a building block for a standardized measure: correlation.