Variance is one of the parameter to describe the distribution. It tells how the distribution extends from the mean value.

Moments are always used to describe the distribution due to its computational simplicity (i.e. the power series). 1st moment is mean value, 2nd moment is variance, 3rd moment is skewness.

Let and denote two random variables. Covariance is a measure of how much and change together.

In this case, variance is a special case of the covariance when two variables are identical.

Given a stochastic process , the autocovariance is the covariance of againsts itself in a time-shifting. That is .

Correlation and Autocorrelation:

Correlation is the measure of the strength of association between two variables. For example, there is a strong correlation between black eyes and and black hair as you may seem them moving together all the time. But, there is a weak correlation between blue wallpaper and puppies. Bear in mind that correlation is not causation. Though the death of babies increase with the cases of vehicle accidents. They have a strong correlation. But we cannot say the death of babies causes the happen of vehicle accidents. It does not make sense. [Taken from Yahoo! Answer]

Autocorrelation is the cross-correlation of the variable itself, in which separated by certain amount of time delay.

Random Variable:

Say we run an experiment in which it gives a finite number of possible outcome, named as sample space .

For the convenience of mathematical calculation, each outcome (denoted as ) of is translated into a real number (if possible) and that translation process is given by a function where . We name as random variable and most of the time we just denote it as if the context is well understood.

Stationary Random Process:

Let denote a random process, in which and denote its mean value and autocorrelation function.

is non-stationary if and vary as varies.

Otherwise, is said stationary or weakly stationary in wide-sense.

For weakly stationary process, and .

For strongly stationary process, all possible moments and join moments are time-invariant.