Taken somewhere from Internet.

Variance, covariance and autocovariance:

  • Variance is one of the parameter to describe the distribution. It tells how the distribution extends from the mean value.
  • Moments are always used to describe the distribution due to its computational simplicity (i.e. the power series). 1st moment is mean value, 2nd moment is variance, 3rd moment is skewness.
  • Let X and Y denote two random variables. Covariance COV(X,Y) is a measure of how much X and Y change together.
  • In this case, variance is a special case of the covariance when two variables are identical.
  • Given a stochastic process X(t), the autocovariance is the covariance of X(t) againsts itself in a time-shifting. That is COV(X(t), X(t-\tau)) .

Correlation and Autocorrelation:

  • Correlation is the measure of the strength of association between two variables. For example, there is a strong correlation between black eyes and and black hair as you may seem them moving together all the time. But, there is a weak correlation between blue wallpaper and puppies. Bear in mind that correlation is not causation. Though the death of babies increase with the cases of vehicle accidents. They have a strong correlation. But we cannot say the death of babies causes the happen of vehicle accidents. It does not make sense. [Taken from Yahoo! Answer]
  • Autocorrelation is the cross-correlation of the variable itself, in which separated by certain amount of time delay.

Random Variable:

  • Say we run an experiment in which it gives a finite number of possible outcome, named as sample space S .
  • For the convenience of mathematical calculation, each outcome (denoted as \zeta ) of S is translated into a real number (if possible) and that translation process is given by a function X(\zeta) where \zeta \in S . We name X(\zeta) as random variable and most of the time we just denote it as X if the context is well understood.

Classification of data.

Stationary Random Process:

  • Let X={x(t)} denote a random process, in which u_x(t) and R_{xx}(t,t+\tau) denote its mean value and autocorrelation function.
  • X is non-stationary if u_x(t) and R_{xx}(t,t+\tau) vary as t varies.
  • Otherwise, X is said stationary or weakly stationary in wide-sense.
  • For weakly stationary process, u_x(t)=u_x and R_{xx}(t,t+\tau)=R_{xx}(\tau).
  • For strongly stationary process, all possible moments and join moments are time-invariant.