Taken somewhere from Internet.
Variance, covariance and autocovariance:
- Variance is one of the parameter to describe the distribution. It tells how the distribution extends from the mean value.
- Moments are always used to describe the distribution due to its computational simplicity (i.e. the power series). 1st moment is mean value, 2nd moment is variance, 3rd moment is skewness.
- Let and denote two random variables. Covariance is a measure of how much and change together.
- In this case, variance is a special case of the covariance when two variables are identical.
- Given a stochastic process , the autocovariance is the covariance of againsts itself in a time-shifting. That is .
Correlation and Autocorrelation:
- Correlation is the measure of the strength of association between two variables. For example, there is a strong correlation between black eyes and and black hair as you may seem them moving together all the time. But, there is a weak correlation between blue wallpaper and puppies. Bear in mind that correlation is not causation. Though the death of babies increase with the cases of vehicle accidents. They have a strong correlation. But we cannot say the death of babies causes the happen of vehicle accidents. It does not make sense. [Taken from Yahoo! Answer]
- Autocorrelation is the cross-correlation of the variable itself, in which separated by certain amount of time delay.
- Say we run an experiment in which it gives a finite number of possible outcome, named as sample space .
- For the convenience of mathematical calculation, each outcome (denoted as ) of is translated into a real number (if possible) and that translation process is given by a function where . We name as random variable and most of the time we just denote it as if the context is well understood.
Stationary Random Process:
- Let denote a random process, in which and denote its mean value and autocorrelation function.
- is non-stationary if and vary as varies.
- Otherwise, is said stationary or weakly stationary in wide-sense.
- For weakly stationary process, and .
- For strongly stationary process, all possible moments and join moments are time-invariant.