Mathematical Probability (STAT2003/STAT7003)
Assignment 3
The due date/time is given on Blackboard. STAT7003 students have additional question(s), marked
with a star (*).
1. Consider the Markov chain {Xn, n = 0, 1, . . .} with transition graph given in Figure 1. Note that,
starting from either state 2 or 3, the chain will eventually reach state 1 or 4 and will stay there
forever. Let T be the first time that either 1 or 4 is hit. For i ∈ {1, 2, 3, 4}, let ai be the probability of
reaching state 1 before 4, starting from state i; that is,
ai = P(XT = 1 | X0 = i), i ∈ {1, 2, 3, 4}.
1 2 3 41
1/3
2/3
1/2
1/2
1
Figure 1: Transition graph of the Markov chain.
(a) Give the one-step transition matrix P for this Markov chain. [1 mark]
(b) Obviously, a1 = 1 and a4 = 0. Find a2 and a3 by conditioning on X1. [2 marks]
(c) Verify your answer in (b) numerically by computing the nth power of P for large n. [1 mark]
(d*) Again by conditioning on X1, find the expected value of T when the process starts from state
2 at time 0. [2 marks]
2. Let X1, X2, . . . be independent standard normal random variables. For a fixed n ∈ {1, 2, . . .}, we
construct a random process on the interval [0, 1], by defining W0 = 0, W1/n = X1/
√
n, and
Wi/n =
X1 + · · · + Xi√
n
, i = 2, . . . , n,
and then we interpolate between the points (0, 0), (1/n,W1/n), . . . , (1,W1). Figure 2 shows a possi-
ble “path” of the process for the case n = 3.
1
0.0 0.2 0.4 0.6 0.8 1.0
t
0.2
0.0
0.2
W
t
Figure 2: A possible outcome of the process (Wt, t ∈ [0, 1]) for n = 3.
(a) Use Python to simulate the process for n = 3 and n = 1000. Make a plots similar to Figure 2.
Python will automatically interpolate between plotted point. [2 marks]
(b) For general n, what is the probability distribution of W1? Justify your answer. [2 marks]
(c) For the case n = 2, what is the probability distribution of W3/4? [2 marks]
(d*) For general n, determine the covariance matrix of W = (W1/n,W2/n, . . . ,Wn/n)>. [2 marks]
3. The continuous random variables X1, X2, . . . are independent and identically distributed (iid) with
pdf f given by
f (x) = cx(1 − x)2, x ∈ [0, 1]
for some normalization constant c. [1 mark]
(a) Find c, showing working.
(b) For large n, the random average (X1+ · · ·+Xn)/n is approximately constant. Give a numerical
value for this constant, using the format x.xxx. Justify your answer. [2 marks]
(c) Approximate, using the central limit theorem, the probability
P(X1 + · · · + X100 > 35).
Write your answer in the format 0.xxxx. [3 marks]
4. The random vector X = [X1, X2, X3]> has a jointly normal distribution with mean vector µ =
[1, 2, 3]> and covariance matrix
Σ =
1 2 42 5 104 10 21
 .
(a) Using the Python function cholesky from numpy.linalg, find a matrix B, such that Σ =
BB>. [1 mark]
(b) Explain how, using the vector µ and matrix B, we can simulate a random vector X from this
multi-variate N(µ,Σ) distribution if we have an algorithm available for simulating standard
normal random variables. [1 mark]
2
(c) Argue why the vector [X1, X2]> has a multi-variate normal distribution and find its mean
vector and covariance matrix. [2 marks]
(d*) Implement a simulation method to sample independent copies of [X1, X2]> in (c), and provide
a scatter plot of 1000 simulated points. [2 marks]
5. Suppose we have a pool of 100 quiz questions. Each quiz is composed of 10 questions, each
randomly selected from the quiz pool. A student can retake the quiz as many times as they like,
though of course eventually they will see questions that they did already before. How many times
should the student have to take the quiz in order to see at least half the question pool with a
probability of at least 0.9?
We can solve this problem via Markov chains. Let Xt be the number of quiz questions seen after
the t-th retake, t = 0, 1, 2, . . ., with X0 = 10.
(a) Suppose that after retake t the student has seen i quiz questions. The number of new questions
k in the next retake can be anything from 0 to the smallest of 10 and the number of remaining
new questions, 100 − i. Derive the one-step probability P(Xt+1 = i + k | Xt = i) in terms of i
and k, where i ∈ {10, . . . 100} and k ∈ {0, . . . ,min(100 − i, 10)}. [1 mark]
(b) The one-step transition matrix P of the Markov chain X1, X2, . . . has 91 rows and 91 columns,
corresponding to the states 10, 11, . . . , 100. The probability that after t retakes at least 50
questions are discovered is
at = Pt(1, 41) + Pt(1, 42) + · · · + Pt(1, 91),
where Pt is the t-th power of the matrix P. Determine (numerically) the numbers a1, . . . , a10
and find out for which t it holds that at ≥ 0.9. Hint: in Python you can use the matrix_power
and comb functions by importing them as follows:
from numpy.linalg import matrix_power
from scipy.special import comb
Be aware that arrays and matrices in Python start with index 0. [3 marks]
6. Consider a moon vehicle with four wheels. The vehicle can function if at least 3 out of the 4
wheels are functioning. And it can even still function with two wheels functioning, as long as the
functioning wheels are not on the same side of the vehicle (i.e., failure of both front, both back,
both left, and both right wheels is not allowed). The wheels are numbered as follows: 1 (front-left),
2 (front-right), 3 (back-left) and 4 (back-right).
(a) Give the structure function of this moon vehicle system with 4 components (wheels) as a truth
table. [1 mark]
(b) What are the minimal cut sets? [1 mark]
(c) Give the structure function by viewing the system as a series of parallel systems. [2 marks]
(d) Show that the reliability function is given by [2 marks]
r(p) = p2p3 + p1p4 − p1p2p3p4.