xuebaunion@vip.163.com

3551 Trousdale Rkwy, University Park, Los Angeles, CA

留学生论文指导和课程辅导

无忧GPA：https://www.essaygpa.com

工作时间：全年无休-早上8点到凌晨3点

微信客服：xiaoxionga100

微信客服：ITCS521

程序代写案例-ST2313

时间：2021-05-05

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

REVISION

University of London

International Programmes

Dr Larry Gui

ST2133 Topics

Module provides a basis for an advanced course in statistical inference.

Aim — to provide a thorough theoretical grounding in probability distributions. The course teaches fundamental material

that is required for specialised courses in statistics, actuarial science and econometrics.

2 Probability space

• Probability measure, Methods for counting, Conditional probability and independence

3 Random variables and univariate distributions

• Random Variables, Distribution functions, Expectation, variance and higher moments.

• Generating functions, Functions of random variables

4 Multivariate distributions

• Joint & marginal distributions, Expectation and Joint Moments , Random vectors and random matrices

• Transformations and Sum of random variables, Multivariate normal distribution

5 Conditional distributions

• Discrete and continuous conditional distributions, Conditional expectation and conditional moments

• Hierarchies and mixtures, Random sums, Conditioning for random vectors

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 2 – PROBABILITY SPACE

University of London

International Programmes

Dr Larry Gui

CHAPTER 2 — Probability space

2.1 Intuitive probability

2.2 Mathematical probability

2.2.1 Measure

2.2.2 Probability measure

2.3 Methods for counting outcomes

2.3.1 Permutations and combinations

2.3.2 Number of combinations and multinomial coefficients

2.4 Conditional probability and independence

2.4.1 Conditional probability

2.4.2 Law of total probability and Bayes’ theorem

2.4.3 Independence

Dr Larry Gui

2.2.1 Measure

Definition 2.2.4

The indicator function for a set

S ⊆ ℝ

IS(x) = {1 if x ∈ S0 otherwise

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 3 – RANDOM VARIABLES AND UNIVARIATE DISTRIBUTIONS

University of London

International Programmes

Dr Larry Gui

Chapter 3 – random variables and univariate distributions

3.1 Random Variables as Functions

3.2 Distribution Functions

3.3 Discrete and continuous random variables

3.4 Expectation, variance and higher moments

3.5 Generating functions

3.6 Functions of random variables

3.7 Sequences of random variables and convergence

Dr Larry Gui

3.4.5 Moments

Definition 3.4.18 (Moments)

For a random variable X and positive integer r, the rth moment of X is

(whenever it is well defined)

Definition 3.4.19 (Central moments)

For a random variable X and positive integer r, the rth central moment of

X is (whenever it is well defined)

μ′ r = E(Xr)

μr = E[(X − E(X ))r]

Dr Larry Gui

3.5.1 Moment generating functions

Definition 3.5.1 (Moment generating function)

The moment generating function of a discrete random variable X is a function

given by

requiring for some h > 0 such that .

Maclaurin expansion for

MX : ℝ → [0,∞)

MX(t) = E(etX)

={

∑x e

tx fX(x) if X is discrete

∫∞−∞ e

tx fX(x)dx if X is continuous

MX(t) <∞ for all t ∈ [−h, h]

ex

ex = 1 + x + 12! x

2 + . . . + 1

r! x

r + . . .

Dr Larry Gui

3.5.1 Moment generating functions

Proposition 3.5.2 (Expansion of Moment generating function)

Generating rth moment (a)

The coefficient of tr in the expansion of the moment generating function is the

rth moment divided by r! That is

MX(t) = E(etX)

1 + tE(X ) + t

2

2! E(X

2) + . . . + t

r

r! E(X

r)

MX(t) = 1 + E(X ) t +

1

2! E(X

2) t2 + . . . + 1

r! E(X

r) tr + . . .

Dr Larry Gui

3.5.1 Moment generating functions

Proposition 3.5.3 (Derivatives of a Moment generating function at zero)

The rth derivative of the moment generating function evaluate at zero is

the rth moment

Proposition 3.5.4 (Uniqueness of Moment generating function)

If X and Y are random variables and we can find h > 0 such that

for all , then .

M(r)X (0) =

dr

dtr

MX(t)

t=0

= E(Xr)

MX(t) = MY(t)

t ∈ [−h, h] FX(x) = FY(x) for all x ∈ ℝ

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 4 – MULTIVARIATE DISTRIBUTIONS

University of London

International Programmes

Dr Larry Gui

Chapter 4 – Multivariate Distributions

4.1 Joint and marginal distributions

4.2 Joint mass and joint density

4.3 Expectation and joint moments

4.4 Independence for pairs of random variables

4.5 Random vectors and random matrices

4.6 Transformations of continuous random variables

4.7 Sums of random variables

4.8 Multivariate normal distribution

Dr Larry Gui

4.2.1 Mass for discrete distributions

Claim 4.2.4 (Marginal mass from joint mass)

For discrete random variables X and Y with joint mass function , the

marginal mass functions are given by

Multivariate extension: for discrete random variables

fX,Y

fX(x) =∑

y

fX,Y(x, y)

fY(y) =∑

x

fX,Y(x, y)

X1, X2, . . . , Xn

fXj(xj) =∑

x1

. . .∑

xj−1

∑

xj+1

. . .∑

xn

fX1,...,Xn(x1, . . . , xn)

Dr Larry Gui

4.2.2 Density for continuous distributions

Definition 4.2.5 (Joint density function)

For jointly continuous random variables X and Y, with joint distribution

function , the joint density function is such that

Claim 4.2.6 (Joint density from joint distribution)

For jointly continuous random variables X and Y, with joint distribution

function , the joint density function is given by

FX,Y fX,Y : ℝ2 → [0,∞)

FX,Y(x, y) = ∫

y

−∞ ∫

x

−∞

fX,Y(u, v)dudv ∀x, y ∈ ℝ

FX,Y

fX,Y(x, y) =

δ2

δuδv

FX,Y(u, v)

u=x,v=y

Dr Larry Gui

4.2.2 Density for continuous distributions

Claim 4.2.9 (Marginal density from joint density)

If X and Y are jointly continuous random variables with joint density

function , then

fX,Y(x, y)

fX(x) = ∫

∞

−∞

fX,Y(x, y)dy

fY(y) = ∫

∞

−∞

fX,Y(x, y)dx

Dr Larry Gui

4.3.2 Covariance and correlation

Definition 4.3.5 (Covariance)

For random variables X and Y, the covariance between X and Y is

defined as

Alternatively,

Cov(X,Y ) = E[(X − E(X ))(Y − E(Y ))]

Cov(X,Y ) = E(XY ) − E(X )E(Y )

Dr Larry Gui

4.6.1 Bivariate transformations

Definition 4.6.1 (Change of variables formula)

If (U, V) is a pair of continuous random variables with support ,

and with g mapping D onto the range , and

, the joint density of X and Y is

where the Jacobian of the inverse transformation is

D ⊆ ℝ2

(X,Y ) = g(U,V ) R ∈ ℝ2

h(X,Y ) = g−1(X,Y ) = (U,V )

fX,Y(x, y) = {fU,V(h(x, y)) |Jh(x, y) | for (x, y) ∈ R0 otherwise

Jh(u, v) =

∂

∂x h1(x, y)

∂

∂x h2(x, y)

∂

∂y h1(x, y)

∂

∂y h2(x, y)

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 5 – CONDITIONAL DISTRIBUTIONS

University of London

International Programmes

Dr Larry Gui

Chapter 5 – Conditional Distributions

5.1 Discrete conditional distributions

5.2 Continuous conditional distributions

5.3 Relationship between joint, marginal and conditional

5.4 Conditional expectation and conditional moments

5.5 Hierarchies and mixtures

5.6 Random sums

5.7 Conditioning for random vectors

Dr Larry Gui

5.2 Continuous conditional distributions

Definition 5.2.1 (Conditional density)

Suppose X and Y are jointly continuous random variables with joint density ,

and the marginal density of X is . The conditional density of Y given is

The distribution function and expected value of are

and

fX,Y

fX X = x

fY|X(y |x) ={

fX,Y(x, y)

fX(x)

for fX(x) > 0

0 otherwise

Y |X = x

FY|X(y |x) = ∫

y

−∞

fY|X(u |x)du

E(Y |X = x) = ∫

∞

−∞

y fY|X(y |x)dy

Dr Larry Gui

5.4 Conditional expectation and moments

Definition 5.4.1 (Conditional expectation)

Suppose X and Y are random variables define

The conditional expectation of is thus which is a

random variable.

Proposition 5.4.2 (Law of iterated expectations)

If X and Y are random variables with conditional expectation ,

ψ (x) = E[Y |X = x] =

∑y y fY|X(y |x) discrete case

∫∞−∞ y fY|X(y |x)dy continuous case

Y |X E(Y |X ) = ψ (X )

E(Y |X )

E(Y ) = E[E(Y |X )]

Dr Larry Gui

5.4.1 Conditional expectation

Proposition 5.4.2 (Law of iterated expectations)

If X and Y are random variables with conditional expectation ,

Consequence:

E(Y |X )

E(Y ) = E[E(Y |X )]

E[Y ] ={

∑x E(Y |X = x) fX(x) discrete case

∫∞−∞E(Y |X = x) fX(x)dx continuous case

Dr Larry Gui

5.4.2 Conditional moments

Lemma 5.4.9 (Useful representation of conditional variance)

For random variables X and Y, the conditional variance of Y given X can

be written as

Proposition 5.4.10 (Decomposition of variance)

For random variables X and Y, the variance of Y is given by

Var(Y |X ) = E(Y2 |X ) − E(Y |X )2

Var(Y ) = E(Var(Y |X )) + Var(E(Y |X ))

Dr Larry Gui

Exam 201b (4.7, 5.6)

3. If X is Gamma distributed with parameters and , i.e. , then it has density:

, for

and for .

(a) If , , and X is independent of Y, derive the distribution of

. You may use the moment generating function of a Gamma random variable without proof, as

long as you state it clearly.

(b) Let be independent of each other and . Each is also

independent of N, which is Poisson distributed with mean µ, so that the probability mass function for

N is given by:

, for n = 0, 1, …

Consider the random variable (where W = 0 if N = 0.)

(i) Derive the moment generating function of W.

(ii) Find the mean of W. You can use the means of a Poisson and a Gamma random variable

without proof. If you use any standard results about sums, you must first state home clearly.

α β X ∼ Gamma(α, β)

fX(x) =

βα

Γ(α) x

α−1e−βx x > 0

Γ(α) = ∫

∞

0

yα−1e−ydy x > 0

X ∼ Gamma(α1, β) Y ∼ Gamma(α2, β)

X + Y

Xi ∼ Gamma(α, β), i = 1,...,N α, β > 0 Xi

pN(n) =

μne−μ

n!

W =

N

∑

i=1

Xi

Section A

Answer all three parts of question 1 (40 marks in total)

1. (a) Let g(x) be a function taking on integer values of x, with

g(x) =

2a, x = −3,−1;

a, x = 0, 2;

3a, x = 1, 3;

0, otherwise.

i. Find a so that g(x) is a probability mass function. [3 marks]

ii. Let X be a discrete random variable with probability mass function g(x).

Find E(X) and Var(X). [5 marks]

iii. Write down the probability mass function of Y = X2− 4|X|+4. [4 marks]

(b) The cumulative distribution function FX(·) for the continuous random variable

X is defined by

FX(x) =

0, x < 0;

ax2/4, 0 ≤ x < 1;

((x− 1)3 + a)/4, 1 ≤ x < 2;

1, x ≥ 2.

i. Find the value of a. [1 mark]

ii. Derive the probability density function of X. [4 marks]

iii. Let W = X2. Derive the cumulative distribution function of W . Hence,

derive the probability density function of W . [7 marks]

(c) Let X follow an exponential distribution with rate λ, i.e., X has a density

function

fX(x) =

{

λe−λx, x > 0;

0, otherwise.

UL18/0327 Page 2 of 7

Section A

Answer all three parts of question 1 (40 marks in total)

1. (a) Let g(x) be a function taking on integer values of x, with

g(x) =

2a, x = −3,−1;

a, x = 0, 2;

3a, x = 1, 3;

0, otherwise.

i. Find a so that g(x) is a probability mass function. [3 marks]

ii. Let X be a discrete random variable with probability mass function g(x).

Find E(X) and Var(X). [5 marks]

iii. Write down the probability mass function of Y = X2− 4|X|+4. [4 marks]

(b) The cumulative distribution function FX(·) for the continuous random variable

X is defined by

FX(x) =

0, x < 0;

ax2/4, 0 ≤ x < 1;

((x− 1)3 + a)/4, 1 ≤ x < 2;

1, x ≥ 2.

i. Find the value of a. [1 mark]

ii. Derive the probability density function of X. [4 marks]

iii. Let W = X2. Derive the cumulative distribution function of W . Hence,

derive the probability density function of W . [7 marks]

(c) Let X follow an exponential distribution with rate λ, i.e., X has a density

function

fX(x) =

{

λe−λx, x > 0;

0, otherwise.

UL18/0327 Page 2 of 7

Section A

Answer all three parts of question 1 (40 marks in total)

1. (a) Let g(x) be a function taking on integer values of x, with

g(x) =

2a, x = −3,−1;

a, x = 0, 2;

3a, x = 1, 3;

0, otherwise.

i. Find a so that g(x) is a probability mass function. [3 marks]

ii. Let X be a discrete random variable with probability mass function g(x).

Find E(X) and Var(X). [5 marks]

iii. Write down the probability mass function of Y = X2− 4|X|+4. [4 marks]

(b) The cumulative distribution function FX(·) for the continuous random variable

X is defined by

FX(x) =

0, x < 0;

ax2/4, 0 ≤ x < 1;

((x− 1)3 + a)/4, 1 ≤ x < 2;

1, x ≥ 2.

i. Find the value of a. [1 mark]

ii. Derive the probability density function of X. [4 marks]

iii. Let W = X2. Derive the cumulative distribution function of W . Hence,

derive the probability density function of W . [7 marks]

(c) Let X follow an exponential distribution with rate λ, i.e., X has a density

function

fX(x) =

{

λe−λx, x > 0;

0, otherwise.

UL18/0327 Page 2 of 7

i. Derive the moment generating function of X. [3 marks]

ii. Let Y be an independent and identically distributed copy of X. For w > 0,

show that

P (X − Y ≤ w) = 1− e

−λw

2

.

(Hint: find the joint density of X and Y first. Determine the valid region

in the double integral involved.) [5 marks]

iii. For w ≤ 0, show that

P (X − Y ≤ w) = e

λw

2

.

[5 marks]

iv. Using parts ii and iii of question (c), show that the density function of

W = X − Y is given by

fW (w) =

λe−λ|w|

2

, w ∈ R.

[3 marks]

UL18/0327 Page 3 of 7

Section B

Answer all three questions in this section (60 marks in total)

2. The conditional density of a random variable X given Y = y is given by

fX|Y (x|y) =

{

x/(2y2), 0 < x < 2y < 2;

0, otherwise.

The conditional density of Y given X = x is given by

fY |X(y|x) =

{

24y2/(8− x3), 0 < x < 2y < 2;

0, otherwise.

(a) Find the ratio fY (y)/fX(x), where fX(x) and fY (y) are the marginal den-

sities of X and Y , respectively. [2 marks]

(b) By integrating out y first in the answer in (a), show that

fX(x) =

{

(5x(8− x3))/48, 0 < x < 2;

0, otherwise.

Is X independent of Y ? Justify your answer. [9 marks]

(c) Let U = XY and V = X/Y . Derive the joint density for U, V , and carefully

state the region for (U, V ) where this joint density is non-zero. [9 marks]

UL18/0327 Page 4 of 7

3. If X is Gamma distributed with parameters α and β, i.e., X ∼ Gamma(α, β),

then it has density

fX(x) =

βα

Γ(α)

xα−1e−βx, x > 0,

and Γ(α) =

∫∞

0 y

α−1e−ydy for α > 0.

(a) Suppose X ∼ Gamma(α1, β1), Y ∼ Gamma(α2, β2), and X is independent

of Y . Derive the distribution of β1X + β2Y . You may use the moment

generating function of a Gamma random variable without proof, as long as

you state it clearly. [7 marks]

(b) Let Xi ∼ Gamma(α, βi), i = 1, . . . , N , be independent of each other and

α, βi > 0. Each Xi is also independent of N , which is Poisson distributed

with mean µ, so that the probability mass function for N is given by

pN(n) =

µne−µ

n!

, n = 0, 1, . . . .

Consider the random variable

W =

N∑

i=1

βiXi,

with the convention that W = 0 if N = 0.

i. Derive the moment generating function of W . [8 marks]

ii. Find the mean of W . You can use the mean of a Poisson random vari-

able without proof. The mean of X ∼ Gamma(α, β) is α/β. [5 marks]

UL18/0327 Page 5 of 7

3. If X is Gamma distributed with parameters α and β, i.e., X ∼ Gamma(α, β),

then it has density

fX(x) =

βα

Γ(α)

xα−1e−βx, x > 0,

and Γ(α) =

∫∞

0 y

α−1e−ydy for α > 0.

(a) Suppose X ∼ Gamma(α1, β1), Y ∼ Gamma(α2, β2), and X is independent

of Y . Derive the distribution of β1X + β2Y . You may use the moment

generating function of a Gamma random variable without proof, as long as

you state it clearly. [7 marks]

(b) Let Xi ∼ Gamma(α, βi), i = 1, . . . , N , be independent of each other and

α, βi > 0. Each Xi is also independent of N , which is Poisson distributed

with mean µ, so that the probability mass function for N is given by

pN(n) =

µne−µ

n!

, n = 0, 1, . . . .

Consider the random variable

W =

N∑

i=1

βiXi,

with the convention that W = 0 if N = 0.

i. Derive the moment generating function of W . [8 marks]

ii. Find the mean of W . You can use the mean of a Poisson random vari-

able without proof. The mean of X ∼ Gamma(α, β) is α/β. [5 marks]

UL18/0327 Page 5 of 7

4. Suppose we have a biased coin, which comes up heads with probability u. An

experiment is carried out so that X is the number of independent flips of the

coin required for r heads to show up, where r ≥ 1 is known.

(a) Show that the probability mass function for X is

pX(x) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

[5 marks]

(b) Suppose U is random and has a density given by

fU(u) =

{

Γ(α+β)

Γ(α)Γ(β)u

α−1(1− u)β−1, 0 < u < 1;

0, otherwise.

where α, β > 0, and Γ(α) is defined in question 3, which has the property

that Γ(α) = (α − 1)Γ(α − 1) for α ≥ 1, and Γ(k) = (k − 1)! for a positive

integer k. The distribution in part (a) thus becomes

pX|U(x|u) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

i. Find the marginal probability mass function of X if α = β = 2.

. [6 marks]

ii. With α = β = 2 still, show that the density of U |X = x is given by

fU |X(u|x) =

{

(x+3)!

(r+1)!(x−r+1)!u

r+1(1− u)x−r+1, 0 < u < 1;

0, otherwise.

Hence find the mean of U |X = x. [5 marks]

(c) Another independent experiment is carried out, with Y denoting the num-

ber of independent flips of the coin required for r heads to show up (the

same r as for the first experiment).

UL18/0327 Page 6 of 7

4. Suppose we have a biased coin, which comes up heads with probability u. An

experiment is carried out so that X is the number of independent flips of the

coin required for r heads to show up, where r ≥ 1 is known.

(a) Show that the probability mass function for X is

pX(x) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

[5 marks]

(b) Suppose U is random and has a density given by

fU(u) =

{

Γ(α+β)

Γ(α)Γ(β)u

α−1(1− u)β−1, 0 < u < 1;

0, otherwise.

where α, β > 0, and Γ(α) is defined in question 3, which has the property

that Γ(α) = (α − 1)Γ(α − 1) for α ≥ 1, and Γ(k) = (k − 1)! for a positive

integer k. The distribution in part (a) thus becomes

pX|U(x|u) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

i. Find the marginal probability mass function of X if α = β = 2.

. [6 marks]

ii. With α = β = 2 still, show that the density of U |X = x is given by

fU |X(u|x) =

{

(x+3)!

(r+1)!(x−r+1)!u

r+1(1− u)x−r+1, 0 < u < 1;

0, otherwise.

Hence find the mean of U |X = x. [5 marks]

(c) Another independent experiment is carried out, with Y denoting the num-

ber of independent flips of the coin required for r heads to show up (the

same r as for the first experiment).

UL18/0327 Page 6 of 7

4. Suppose we have a biased coin, which comes up heads with probability u. An

experiment is carried out so that X is the number of independent flips of the

coin required for r heads to show up, where r ≥ 1 is known.

(a) Show that the probability mass function for X is

pX(x) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

[5 marks]

(b) Suppose U is random and has a density given by

fU(u) =

{

Γ(α+β)

Γ(α)Γ(β)u

α−1(1− u)β−1, 0 < u < 1;

0, otherwise.

where α, β > 0, and Γ(α) is defined in question 3, which has the property

that Γ(α) = (α − 1)Γ(α − 1) for α ≥ 1, and Γ(k) = (k − 1)! for a positive

integer k. The distribution in part (a) thus becomes

pX|U(x|u) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

i. Find the marginal probability mass function of X if α = β = 2.

. [6 marks]

ii. With α = β = 2 still, show that the density of U |X = x is given by

fU |X(u|x) =

{

(x+3)!

(r+1)!(x−r+1)!u

r+1(1− u)x−r+1, 0 < u < 1;

0, otherwise.

Hence find the mean of U |X = x. [5 marks]

(c) Another independent experiment is carried out, with Y denoting the num-

ber of independent flips of the coin required for r heads to show up (the

same r as for the first experiment).

UL18/0327 Page 6 of 7

State (no need for a derivation) the density of U |(X, Y ) = (x, y) and its

mean, where U still has the density in part (b) with α = β = 2. [4 marks]

END OF PAPER

UL18/0327 Page 7 of 7

学霸联盟

ADVANCED STATISTICS — DISTRIBUTION THEORY

REVISION

University of London

International Programmes

Dr Larry Gui

ST2133 Topics

Module provides a basis for an advanced course in statistical inference.

Aim — to provide a thorough theoretical grounding in probability distributions. The course teaches fundamental material

that is required for specialised courses in statistics, actuarial science and econometrics.

2 Probability space

• Probability measure, Methods for counting, Conditional probability and independence

3 Random variables and univariate distributions

• Random Variables, Distribution functions, Expectation, variance and higher moments.

• Generating functions, Functions of random variables

4 Multivariate distributions

• Joint & marginal distributions, Expectation and Joint Moments , Random vectors and random matrices

• Transformations and Sum of random variables, Multivariate normal distribution

5 Conditional distributions

• Discrete and continuous conditional distributions, Conditional expectation and conditional moments

• Hierarchies and mixtures, Random sums, Conditioning for random vectors

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 2 – PROBABILITY SPACE

University of London

International Programmes

Dr Larry Gui

CHAPTER 2 — Probability space

2.1 Intuitive probability

2.2 Mathematical probability

2.2.1 Measure

2.2.2 Probability measure

2.3 Methods for counting outcomes

2.3.1 Permutations and combinations

2.3.2 Number of combinations and multinomial coefficients

2.4 Conditional probability and independence

2.4.1 Conditional probability

2.4.2 Law of total probability and Bayes’ theorem

2.4.3 Independence

Dr Larry Gui

2.2.1 Measure

Definition 2.2.4

The indicator function for a set

S ⊆ ℝ

IS(x) = {1 if x ∈ S0 otherwise

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 3 – RANDOM VARIABLES AND UNIVARIATE DISTRIBUTIONS

University of London

International Programmes

Dr Larry Gui

Chapter 3 – random variables and univariate distributions

3.1 Random Variables as Functions

3.2 Distribution Functions

3.3 Discrete and continuous random variables

3.4 Expectation, variance and higher moments

3.5 Generating functions

3.6 Functions of random variables

3.7 Sequences of random variables and convergence

Dr Larry Gui

3.4.5 Moments

Definition 3.4.18 (Moments)

For a random variable X and positive integer r, the rth moment of X is

(whenever it is well defined)

Definition 3.4.19 (Central moments)

For a random variable X and positive integer r, the rth central moment of

X is (whenever it is well defined)

μ′ r = E(Xr)

μr = E[(X − E(X ))r]

Dr Larry Gui

3.5.1 Moment generating functions

Definition 3.5.1 (Moment generating function)

The moment generating function of a discrete random variable X is a function

given by

requiring for some h > 0 such that .

Maclaurin expansion for

MX : ℝ → [0,∞)

MX(t) = E(etX)

={

∑x e

tx fX(x) if X is discrete

∫∞−∞ e

tx fX(x)dx if X is continuous

MX(t) <∞ for all t ∈ [−h, h]

ex

ex = 1 + x + 12! x

2 + . . . + 1

r! x

r + . . .

Dr Larry Gui

3.5.1 Moment generating functions

Proposition 3.5.2 (Expansion of Moment generating function)

Generating rth moment (a)

The coefficient of tr in the expansion of the moment generating function is the

rth moment divided by r! That is

MX(t) = E(etX)

1 + tE(X ) + t

2

2! E(X

2) + . . . + t

r

r! E(X

r)

MX(t) = 1 + E(X ) t +

1

2! E(X

2) t2 + . . . + 1

r! E(X

r) tr + . . .

Dr Larry Gui

3.5.1 Moment generating functions

Proposition 3.5.3 (Derivatives of a Moment generating function at zero)

The rth derivative of the moment generating function evaluate at zero is

the rth moment

Proposition 3.5.4 (Uniqueness of Moment generating function)

If X and Y are random variables and we can find h > 0 such that

for all , then .

M(r)X (0) =

dr

dtr

MX(t)

t=0

= E(Xr)

MX(t) = MY(t)

t ∈ [−h, h] FX(x) = FY(x) for all x ∈ ℝ

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 4 – MULTIVARIATE DISTRIBUTIONS

University of London

International Programmes

Dr Larry Gui

Chapter 4 – Multivariate Distributions

4.1 Joint and marginal distributions

4.2 Joint mass and joint density

4.3 Expectation and joint moments

4.4 Independence for pairs of random variables

4.5 Random vectors and random matrices

4.6 Transformations of continuous random variables

4.7 Sums of random variables

4.8 Multivariate normal distribution

Dr Larry Gui

4.2.1 Mass for discrete distributions

Claim 4.2.4 (Marginal mass from joint mass)

For discrete random variables X and Y with joint mass function , the

marginal mass functions are given by

Multivariate extension: for discrete random variables

fX,Y

fX(x) =∑

y

fX,Y(x, y)

fY(y) =∑

x

fX,Y(x, y)

X1, X2, . . . , Xn

fXj(xj) =∑

x1

. . .∑

xj−1

∑

xj+1

. . .∑

xn

fX1,...,Xn(x1, . . . , xn)

Dr Larry Gui

4.2.2 Density for continuous distributions

Definition 4.2.5 (Joint density function)

For jointly continuous random variables X and Y, with joint distribution

function , the joint density function is such that

Claim 4.2.6 (Joint density from joint distribution)

For jointly continuous random variables X and Y, with joint distribution

function , the joint density function is given by

FX,Y fX,Y : ℝ2 → [0,∞)

FX,Y(x, y) = ∫

y

−∞ ∫

x

−∞

fX,Y(u, v)dudv ∀x, y ∈ ℝ

FX,Y

fX,Y(x, y) =

δ2

δuδv

FX,Y(u, v)

u=x,v=y

Dr Larry Gui

4.2.2 Density for continuous distributions

Claim 4.2.9 (Marginal density from joint density)

If X and Y are jointly continuous random variables with joint density

function , then

fX,Y(x, y)

fX(x) = ∫

∞

−∞

fX,Y(x, y)dy

fY(y) = ∫

∞

−∞

fX,Y(x, y)dx

Dr Larry Gui

4.3.2 Covariance and correlation

Definition 4.3.5 (Covariance)

For random variables X and Y, the covariance between X and Y is

defined as

Alternatively,

Cov(X,Y ) = E[(X − E(X ))(Y − E(Y ))]

Cov(X,Y ) = E(XY ) − E(X )E(Y )

Dr Larry Gui

4.6.1 Bivariate transformations

Definition 4.6.1 (Change of variables formula)

If (U, V) is a pair of continuous random variables with support ,

and with g mapping D onto the range , and

, the joint density of X and Y is

where the Jacobian of the inverse transformation is

D ⊆ ℝ2

(X,Y ) = g(U,V ) R ∈ ℝ2

h(X,Y ) = g−1(X,Y ) = (U,V )

fX,Y(x, y) = {fU,V(h(x, y)) |Jh(x, y) | for (x, y) ∈ R0 otherwise

Jh(u, v) =

∂

∂x h1(x, y)

∂

∂x h2(x, y)

∂

∂y h1(x, y)

∂

∂y h2(x, y)

ST2313

ADVANCED STATISTICS — DISTRIBUTION THEORY

CHAPTER 5 – CONDITIONAL DISTRIBUTIONS

University of London

International Programmes

Dr Larry Gui

Chapter 5 – Conditional Distributions

5.1 Discrete conditional distributions

5.2 Continuous conditional distributions

5.3 Relationship between joint, marginal and conditional

5.4 Conditional expectation and conditional moments

5.5 Hierarchies and mixtures

5.6 Random sums

5.7 Conditioning for random vectors

Dr Larry Gui

5.2 Continuous conditional distributions

Definition 5.2.1 (Conditional density)

Suppose X and Y are jointly continuous random variables with joint density ,

and the marginal density of X is . The conditional density of Y given is

The distribution function and expected value of are

and

fX,Y

fX X = x

fY|X(y |x) ={

fX,Y(x, y)

fX(x)

for fX(x) > 0

0 otherwise

Y |X = x

FY|X(y |x) = ∫

y

−∞

fY|X(u |x)du

E(Y |X = x) = ∫

∞

−∞

y fY|X(y |x)dy

Dr Larry Gui

5.4 Conditional expectation and moments

Definition 5.4.1 (Conditional expectation)

Suppose X and Y are random variables define

The conditional expectation of is thus which is a

random variable.

Proposition 5.4.2 (Law of iterated expectations)

If X and Y are random variables with conditional expectation ,

ψ (x) = E[Y |X = x] =

∑y y fY|X(y |x) discrete case

∫∞−∞ y fY|X(y |x)dy continuous case

Y |X E(Y |X ) = ψ (X )

E(Y |X )

E(Y ) = E[E(Y |X )]

Dr Larry Gui

5.4.1 Conditional expectation

Proposition 5.4.2 (Law of iterated expectations)

If X and Y are random variables with conditional expectation ,

Consequence:

E(Y |X )

E(Y ) = E[E(Y |X )]

E[Y ] ={

∑x E(Y |X = x) fX(x) discrete case

∫∞−∞E(Y |X = x) fX(x)dx continuous case

Dr Larry Gui

5.4.2 Conditional moments

Lemma 5.4.9 (Useful representation of conditional variance)

For random variables X and Y, the conditional variance of Y given X can

be written as

Proposition 5.4.10 (Decomposition of variance)

For random variables X and Y, the variance of Y is given by

Var(Y |X ) = E(Y2 |X ) − E(Y |X )2

Var(Y ) = E(Var(Y |X )) + Var(E(Y |X ))

Dr Larry Gui

Exam 201b (4.7, 5.6)

3. If X is Gamma distributed with parameters and , i.e. , then it has density:

, for

and for .

(a) If , , and X is independent of Y, derive the distribution of

. You may use the moment generating function of a Gamma random variable without proof, as

long as you state it clearly.

(b) Let be independent of each other and . Each is also

independent of N, which is Poisson distributed with mean µ, so that the probability mass function for

N is given by:

, for n = 0, 1, …

Consider the random variable (where W = 0 if N = 0.)

(i) Derive the moment generating function of W.

(ii) Find the mean of W. You can use the means of a Poisson and a Gamma random variable

without proof. If you use any standard results about sums, you must first state home clearly.

α β X ∼ Gamma(α, β)

fX(x) =

βα

Γ(α) x

α−1e−βx x > 0

Γ(α) = ∫

∞

0

yα−1e−ydy x > 0

X ∼ Gamma(α1, β) Y ∼ Gamma(α2, β)

X + Y

Xi ∼ Gamma(α, β), i = 1,...,N α, β > 0 Xi

pN(n) =

μne−μ

n!

W =

N

∑

i=1

Xi

Section A

Answer all three parts of question 1 (40 marks in total)

1. (a) Let g(x) be a function taking on integer values of x, with

g(x) =

2a, x = −3,−1;

a, x = 0, 2;

3a, x = 1, 3;

0, otherwise.

i. Find a so that g(x) is a probability mass function. [3 marks]

ii. Let X be a discrete random variable with probability mass function g(x).

Find E(X) and Var(X). [5 marks]

iii. Write down the probability mass function of Y = X2− 4|X|+4. [4 marks]

(b) The cumulative distribution function FX(·) for the continuous random variable

X is defined by

FX(x) =

0, x < 0;

ax2/4, 0 ≤ x < 1;

((x− 1)3 + a)/4, 1 ≤ x < 2;

1, x ≥ 2.

i. Find the value of a. [1 mark]

ii. Derive the probability density function of X. [4 marks]

iii. Let W = X2. Derive the cumulative distribution function of W . Hence,

derive the probability density function of W . [7 marks]

(c) Let X follow an exponential distribution with rate λ, i.e., X has a density

function

fX(x) =

{

λe−λx, x > 0;

0, otherwise.

UL18/0327 Page 2 of 7

Section A

Answer all three parts of question 1 (40 marks in total)

1. (a) Let g(x) be a function taking on integer values of x, with

g(x) =

2a, x = −3,−1;

a, x = 0, 2;

3a, x = 1, 3;

0, otherwise.

i. Find a so that g(x) is a probability mass function. [3 marks]

ii. Let X be a discrete random variable with probability mass function g(x).

Find E(X) and Var(X). [5 marks]

iii. Write down the probability mass function of Y = X2− 4|X|+4. [4 marks]

(b) The cumulative distribution function FX(·) for the continuous random variable

X is defined by

FX(x) =

0, x < 0;

ax2/4, 0 ≤ x < 1;

((x− 1)3 + a)/4, 1 ≤ x < 2;

1, x ≥ 2.

i. Find the value of a. [1 mark]

ii. Derive the probability density function of X. [4 marks]

iii. Let W = X2. Derive the cumulative distribution function of W . Hence,

derive the probability density function of W . [7 marks]

(c) Let X follow an exponential distribution with rate λ, i.e., X has a density

function

fX(x) =

{

λe−λx, x > 0;

0, otherwise.

UL18/0327 Page 2 of 7

Section A

Answer all three parts of question 1 (40 marks in total)

1. (a) Let g(x) be a function taking on integer values of x, with

g(x) =

2a, x = −3,−1;

a, x = 0, 2;

3a, x = 1, 3;

0, otherwise.

i. Find a so that g(x) is a probability mass function. [3 marks]

ii. Let X be a discrete random variable with probability mass function g(x).

Find E(X) and Var(X). [5 marks]

iii. Write down the probability mass function of Y = X2− 4|X|+4. [4 marks]

(b) The cumulative distribution function FX(·) for the continuous random variable

X is defined by

FX(x) =

0, x < 0;

ax2/4, 0 ≤ x < 1;

((x− 1)3 + a)/4, 1 ≤ x < 2;

1, x ≥ 2.

i. Find the value of a. [1 mark]

ii. Derive the probability density function of X. [4 marks]

iii. Let W = X2. Derive the cumulative distribution function of W . Hence,

derive the probability density function of W . [7 marks]

(c) Let X follow an exponential distribution with rate λ, i.e., X has a density

function

fX(x) =

{

λe−λx, x > 0;

0, otherwise.

UL18/0327 Page 2 of 7

i. Derive the moment generating function of X. [3 marks]

ii. Let Y be an independent and identically distributed copy of X. For w > 0,

show that

P (X − Y ≤ w) = 1− e

−λw

2

.

(Hint: find the joint density of X and Y first. Determine the valid region

in the double integral involved.) [5 marks]

iii. For w ≤ 0, show that

P (X − Y ≤ w) = e

λw

2

.

[5 marks]

iv. Using parts ii and iii of question (c), show that the density function of

W = X − Y is given by

fW (w) =

λe−λ|w|

2

, w ∈ R.

[3 marks]

UL18/0327 Page 3 of 7

Section B

Answer all three questions in this section (60 marks in total)

2. The conditional density of a random variable X given Y = y is given by

fX|Y (x|y) =

{

x/(2y2), 0 < x < 2y < 2;

0, otherwise.

The conditional density of Y given X = x is given by

fY |X(y|x) =

{

24y2/(8− x3), 0 < x < 2y < 2;

0, otherwise.

(a) Find the ratio fY (y)/fX(x), where fX(x) and fY (y) are the marginal den-

sities of X and Y , respectively. [2 marks]

(b) By integrating out y first in the answer in (a), show that

fX(x) =

{

(5x(8− x3))/48, 0 < x < 2;

0, otherwise.

Is X independent of Y ? Justify your answer. [9 marks]

(c) Let U = XY and V = X/Y . Derive the joint density for U, V , and carefully

state the region for (U, V ) where this joint density is non-zero. [9 marks]

UL18/0327 Page 4 of 7

3. If X is Gamma distributed with parameters α and β, i.e., X ∼ Gamma(α, β),

then it has density

fX(x) =

βα

Γ(α)

xα−1e−βx, x > 0,

and Γ(α) =

∫∞

0 y

α−1e−ydy for α > 0.

(a) Suppose X ∼ Gamma(α1, β1), Y ∼ Gamma(α2, β2), and X is independent

of Y . Derive the distribution of β1X + β2Y . You may use the moment

generating function of a Gamma random variable without proof, as long as

you state it clearly. [7 marks]

(b) Let Xi ∼ Gamma(α, βi), i = 1, . . . , N , be independent of each other and

α, βi > 0. Each Xi is also independent of N , which is Poisson distributed

with mean µ, so that the probability mass function for N is given by

pN(n) =

µne−µ

n!

, n = 0, 1, . . . .

Consider the random variable

W =

N∑

i=1

βiXi,

with the convention that W = 0 if N = 0.

i. Derive the moment generating function of W . [8 marks]

ii. Find the mean of W . You can use the mean of a Poisson random vari-

able without proof. The mean of X ∼ Gamma(α, β) is α/β. [5 marks]

UL18/0327 Page 5 of 7

3. If X is Gamma distributed with parameters α and β, i.e., X ∼ Gamma(α, β),

then it has density

fX(x) =

βα

Γ(α)

xα−1e−βx, x > 0,

and Γ(α) =

∫∞

0 y

α−1e−ydy for α > 0.

(a) Suppose X ∼ Gamma(α1, β1), Y ∼ Gamma(α2, β2), and X is independent

of Y . Derive the distribution of β1X + β2Y . You may use the moment

generating function of a Gamma random variable without proof, as long as

you state it clearly. [7 marks]

(b) Let Xi ∼ Gamma(α, βi), i = 1, . . . , N , be independent of each other and

α, βi > 0. Each Xi is also independent of N , which is Poisson distributed

with mean µ, so that the probability mass function for N is given by

pN(n) =

µne−µ

n!

, n = 0, 1, . . . .

Consider the random variable

W =

N∑

i=1

βiXi,

with the convention that W = 0 if N = 0.

i. Derive the moment generating function of W . [8 marks]

ii. Find the mean of W . You can use the mean of a Poisson random vari-

able without proof. The mean of X ∼ Gamma(α, β) is α/β. [5 marks]

UL18/0327 Page 5 of 7

4. Suppose we have a biased coin, which comes up heads with probability u. An

experiment is carried out so that X is the number of independent flips of the

coin required for r heads to show up, where r ≥ 1 is known.

(a) Show that the probability mass function for X is

pX(x) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

[5 marks]

(b) Suppose U is random and has a density given by

fU(u) =

{

Γ(α+β)

Γ(α)Γ(β)u

α−1(1− u)β−1, 0 < u < 1;

0, otherwise.

where α, β > 0, and Γ(α) is defined in question 3, which has the property

that Γ(α) = (α − 1)Γ(α − 1) for α ≥ 1, and Γ(k) = (k − 1)! for a positive

integer k. The distribution in part (a) thus becomes

pX|U(x|u) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

i. Find the marginal probability mass function of X if α = β = 2.

. [6 marks]

ii. With α = β = 2 still, show that the density of U |X = x is given by

fU |X(u|x) =

{

(x+3)!

(r+1)!(x−r+1)!u

r+1(1− u)x−r+1, 0 < u < 1;

0, otherwise.

Hence find the mean of U |X = x. [5 marks]

(c) Another independent experiment is carried out, with Y denoting the num-

ber of independent flips of the coin required for r heads to show up (the

same r as for the first experiment).

UL18/0327 Page 6 of 7

4. Suppose we have a biased coin, which comes up heads with probability u. An

experiment is carried out so that X is the number of independent flips of the

coin required for r heads to show up, where r ≥ 1 is known.

(a) Show that the probability mass function for X is

pX(x) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

[5 marks]

(b) Suppose U is random and has a density given by

fU(u) =

{

Γ(α+β)

Γ(α)Γ(β)u

α−1(1− u)β−1, 0 < u < 1;

0, otherwise.

where α, β > 0, and Γ(α) is defined in question 3, which has the property

that Γ(α) = (α − 1)Γ(α − 1) for α ≥ 1, and Γ(k) = (k − 1)! for a positive

integer k. The distribution in part (a) thus becomes

pX|U(x|u) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

i. Find the marginal probability mass function of X if α = β = 2.

. [6 marks]

ii. With α = β = 2 still, show that the density of U |X = x is given by

fU |X(u|x) =

{

(x+3)!

(r+1)!(x−r+1)!u

r+1(1− u)x−r+1, 0 < u < 1;

0, otherwise.

Hence find the mean of U |X = x. [5 marks]

(c) Another independent experiment is carried out, with Y denoting the num-

ber of independent flips of the coin required for r heads to show up (the

same r as for the first experiment).

UL18/0327 Page 6 of 7

4. Suppose we have a biased coin, which comes up heads with probability u. An

experiment is carried out so that X is the number of independent flips of the

coin required for r heads to show up, where r ≥ 1 is known.

(a) Show that the probability mass function for X is

pX(x) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

[5 marks]

(b) Suppose U is random and has a density given by

fU(u) =

{

Γ(α+β)

Γ(α)Γ(β)u

α−1(1− u)β−1, 0 < u < 1;

0, otherwise.

where α, β > 0, and Γ(α) is defined in question 3, which has the property

that Γ(α) = (α − 1)Γ(α − 1) for α ≥ 1, and Γ(k) = (k − 1)! for a positive

integer k. The distribution in part (a) thus becomes

pX|U(x|u) =

{ (

x−1

r−1

)

ur(1− u)x−r, x = r, r + 1, . . . .;

0, otherwise.

i. Find the marginal probability mass function of X if α = β = 2.

. [6 marks]

ii. With α = β = 2 still, show that the density of U |X = x is given by

fU |X(u|x) =

{

(x+3)!

(r+1)!(x−r+1)!u

r+1(1− u)x−r+1, 0 < u < 1;

0, otherwise.

Hence find the mean of U |X = x. [5 marks]

(c) Another independent experiment is carried out, with Y denoting the num-

ber of independent flips of the coin required for r heads to show up (the

same r as for the first experiment).

UL18/0327 Page 6 of 7

State (no need for a derivation) the density of U |(X, Y ) = (x, y) and its

mean, where U still has the density in part (b) with α = β = 2. [4 marks]

END OF PAPER

UL18/0327 Page 7 of 7

学霸联盟