# Moment Generating Function

Published:

As the name suggests, moment generating function (MGF) provides a function that generates moments, such as `E[X]`, `E[X^2]`, `E[X^3]`, and so forth.

MGF formula is given by the following.

``````MGFx(t) = E[e^(t.X)] where X is a random variable
``````

Since `E[e^(t.X)]` simply means `(e^(t.x1)) . P(x1) + (e^(t.x2)) . P(x2) + (e^(t.x3)) . P(x3) + ... + (e^(t.xn)) . P(xn)`, we can generate the respective formula for discrete and continuous random variable, such as the following.

``````Discrete RV

MGFx(t) = SUM(i=1 to n) (e^(t.xi)) . P(xi)

Continuous RV

MGFx(t) = INTEGRAL(-inf to inf) (e^(t.x)) . PDF(x) dx
``````

After performing the above `MGFx(t)` calculation, we can generate the moments by taking its derivative with respect to `t` and evaluated at `t = 0` like the following.

``````MGFx’(t) evaluated at t = 0 => E[X]
MGFx’’(t) evaluated at t = 0 => E[X^2]
MGFx’’’(t) evaluated at t = 0 => E[X^3]
.
.
.
``````

## Proof

We’re gonna prove that taking the n-th derivative of the MGF (respect to `t` and evaluated at `t = 0`) results in `E[X^n]`.

Recall that the Taylor series of exponential function is given below.

``````e^x = SUM(n = 0 to inf) (x^n) / n! = 1 + x + (x^2) / 2! + (x^3) / 3! + ...
``````

Applying the above Taylor series to `e^(t.X)` yields the following.

``````e^(t.X) = [1 + t.X + ((tX)^2) / 2! + ((tX)^3) / 3! + ...]
``````

Taking the expected value of the above.

``````E[e^(t.X)] = E[1 + t.X + ((tX)^2) / 2! + ((tX)^3) / 3! + ...]

E[e^(t.X)] = 1 + t . E[X] + (t^2 / 2!) . E[X^2] + (t^3 / 3!) . E[X^3] + ...
``````

We already have the MGF represented in the Taylor series of `e^(t.X)`.

Take a look at what happens when we take the derivatives with respect to `t`.

### 1st derivative

``````MGFx’(t) = E[X] + 2 . t / 2! . E[X^2] + 3 . t^2 / 3! . E[X^3] + ...
``````

Evaluating the above at `t = 0` yields the following.

``````MGFx’(0) = E[X]
``````

### 2nd derivative

``````MGFx’’(t) = 2 / 2! . E[X^2] + 3 . 2 . t / 3! . E[X^3] + ...
``````

Evaluating the above at `t = 0` yields the following.

``````MGFx’’(0) = E[X^2]
``````

### 3rd derivative

``````MGFx’’’(t) = 3 . 2 / 3! . E[X^3] + ...
``````

Evaluating the above at `t = 0` yields the following.

``````MGFx’’’(0) = E[X^3]
``````

### n-th derivative

``````MGFx^n(t) = n . (n-1) . (n-2) ... 1 / n! . E[X^n] + ...
``````

Evaluating the above at `t = 0` yields the following.

``````MGFx^n(0) = E[X^n]
``````

## Final Note

For centralized moment, such as `E[X-mu]`, `E[(X-mu)^2]`, `E[(X-mu)^3]` and so on, we can just replace the `X` in the MGF formula with `X-mu`, such as `E[e^(t.(X-mu))]`. The rests should be the same.

Tags: