Demystify Convolution

March 18, 2011March 18, 2011 ~ zkchong ~ Leave a comment

There are may ways to explain convolution. But I prefer this example.

# 1

Imagine you are throwing a stone into a lake.

As the stone hit to the lake, you will see some ripples on the surface of the lake.

That is the response of the lake towards your projection of stone.

# 2

Now, let say we want to model this.

Your stone is an unit impulse ( $\delta(t)$ ) and the lake is a transfer function $h(t)$ .

We don’t what exactly is $h(t)$ yet.

However, when your stone hits into the lake , we see a ripple or response . That ripple is the response of the lake towards your stone.

Or, mathematically, $\delta(t)$ convolute with $h(t)$ and we see an output something like the figure below.

(Of cause, the response of the lake is not a pure rectangular wave form. This is just for our ease of illustration.)

Continue reading “Demystify Convolution” →

The Random Variables and Random Process

January 31, 2011 ~ zkchong ~ 6 Comments

small_wooden_bucket_with_apples — Bucket of Apples

Someone ask me to explain the different between random variables and random process. Here are my explanation.

Say we are a wholesaler of apples in a city. One day a worker moves down a bucket of apples from a truck. By looking at the apples in this bucket, we can measure the expected weight and variation of apples in this bucket. Let denote the expected weight as $X$ , in which it is a random variable that possesses finite mean and variation.

Then, the worker moves down another bucket of apples. We not sure if this bucket of apples possess the same statistical properties as the previous bucket. By intuition, we know these apples in the second bucket possesses a finite expected weight as well (or you will see a HUGE apple!!). Next, the worker continue to move down a lot of buckets of apples from the truck (big truck!). For convenience, we denote the expected weight as $X(t)$ where $t=1,2,...$ denotes the sequence where the worker moves down the buckets.

The manager of the wholesaler is not interested at the weight distribution of apples in any particular bucket but among certain groups of buckets. For example, he wants to know if the first bucket has worm, what is the chance that the nth bucket has worm as well. This question the inter-relation among the buckets. So, we have no choice but to extend the notion “a single bucket” into “groups of buckets”. In mathematics, we write this as $\{X(t):t=1,2,\dots \}$ and call this the random process (or stochastic process) of X. In such a way, we can describe the correlation (or other statistical properties) between the first bucket and the second bucket or any bucket in the later sequence to the manager.

As for the summary, in random process (stochastic process) we are dealing with a group of “homogeneous” random variables in the sense that they are of the same function (e.g. apples) but they may possess different distribution function.

[Math] My Math Note

October 20, 2010January 21, 2011 ~ zkchong ~ Leave a comment

Taken somewhere from Internet.

Variance, covariance and autocovariance:

Variance is one of the parameter to describe the distribution. It tells how the distribution extends from the mean value.
Moments are always used to describe the distribution due to its computational simplicity (i.e. the power series). 1st moment is mean value, 2nd moment is variance, 3rd moment is skewness.
Let $X$ and $Y$ denote two random variables. Covariance $COV(X,Y)$ is a measure of how much $X$ and $Y$ change together.
In this case, variance is a special case of the covariance when two variables are identical.
Given a stochastic process $X(t)$ , the autocovariance is the covariance of $X(t)$ againsts itself in a time-shifting. That is $COV(X(t), X(t-\tau))$ .

Correlation and Autocorrelation:

Correlation is the measure of the strength of association between two variables. For example, there is a strong correlation between black eyes and and black hair as you may seem them moving together all the time. But, there is a weak correlation between blue wallpaper and puppies. Bear in mind that correlation is not causation. Though the death of babies increase with the cases of vehicle accidents. They have a strong correlation. But we cannot say the death of babies causes the happen of vehicle accidents. It does not make sense. [Taken from Yahoo! Answer]
Autocorrelation is the cross-correlation of the variable itself, in which separated by certain amount of time delay.

Random Variable:

Say we run an experiment in which it gives a finite number of possible outcome, named as sample space $S$ .
For the convenience of mathematical calculation, each outcome (denoted as $\zeta$ ) of $S$ is translated into a real number (if possible) and that translation process is given by a function $X(\zeta)$ where $\zeta \in S$ . We name $X(\zeta)$ as random variable and most of the time we just denote it as $X$ if the context is well understood.

Stationary Random Process:

Let $X={x(t)}$ denote a random process, in which $u_x(t)$ and $R_{xx}(t,t+\tau)$ denote its mean value and autocorrelation function.
$X$ is non-stationary if $u_x(t)$ and $R_{xx}(t,t+\tau)$ vary as $t$ varies.
Otherwise, $X$ is said stationary or weakly stationary in wide-sense.
For weakly stationary process, $u_x(t)=u_x$ and $R_{xx}(t,t+\tau)=R_{xx}(\tau)$ .
For strongly stationary process, all possible moments and join moments are time-invariant.