Covariance and Expectation

Problem

Covariance provides a measure of the strength of correlation between two or more random variables. For two random variables, the covariance is a measure of the relationship between the two random variables and to what extent they change together.

The covariance of two random variables $X, Y$ is defined using expectation

$cov(X,Y) = E[(X - E[X])(Y - E[Y])]$.

Part 1: Show that $cov(X,Y) = E[XY] - E[X]E[Y]$.

Part 2: Show that $cov(X,X) = Var(X)$.

Solution

Part 1

$cov(X,Y) = E[(X - E[X])(Y - E[Y])]$

$cov(X,Y) = E[XY - X(E[Y]) - Y(E[X]) + E[X]E[Y]]$

$cov(X,Y) = E[XY] - E[X(E[Y])] - E[Y(E[X])] + E[E[X]E[Y]]$

Since $E[X]$ and $E[Y]$ are constants,

$cov(X,Y) = E[XY] - E[X]E[Y] - E[X]E[Y] + E[X]E[Y]E[1]$

$cov(X,Y) = E[XY] - E[X]E[Y] - E[X]E[Y] + E[X]E[Y]$

$cov(X,Y) = E[XY] - E[X]E[Y]$.

Part 2

Knowing that $Var(X) = E[X^2] - {(E[X])}^2$ (proof here), it is obvious from the result in part 1 that

$cov(X,Y) = E[XX] - E[X]E[X]$

$cov(X,Y) = E[X^2] - {(E[X])}^2$

$cov(X,X) = Var(X)$.