MATH 337: Assignment 2. Due: February 22, 2021
1. (a) If an is a sequence of real numbers such that an → a as n tends
to infinity, show that
1
n
n∑
k=1
ak → a,
as n tends to infinity.
2. (a) For the 4-state Markov process whose transition matrix is given
by 
1/2 1/2 0 0
2/3 0 0 1/3
0 0 4/5 1/5
1 0 0 0
 ,
draw the associated directed graph and identify which states are
recurrent, transient and absorbing.
(b) Determine if the process is irreducible and if so, is it aperiodic?
3. Labelling the rows of the matrix in 2(a) as corresponding to the
states 0, 1, 2, and 3 respectively, calculate the probabilities f01, f11,
f21 and f31 for the 4-state Markov process in Question 2.
4. (a) If P is the transition matrix of an irreducible Markov chain with
finitely many states, show that Q := (I + P )/2, where I is the
identity matrix, is also a transition matrix corresponding to an
irreducible Markov chain.
(b) Show that P and Q have the same stationary distribution.
5. (a) Consider the Markov process with transition matrix P given by
P =
1/2 1/3 1/63/4 0 1/4
0 1 0
 .
Show that the process is irreducible and aperiodic.
(b) Calculate the stationary probabiiity distribution for the process
in (a).
26. A transition matrix P is said to be doubly stochastic if the sum
over each column equals 1; that is,
M∑
i=0
pij = 1 ∀j.
If the Markov chain described by P is irreducible, aperiodic and
consists of M + 1 states, show that the stationary distribution is
given by
pij =
1
M + 1
, for j = 0, 1, ...,M.
7. A fair dice (with six faces, numbered from 1 to 6) is rolled repeat-
edly and independently. Show that the mean recurrence time for
any given number is 6.
8. If A and B are M ×M row stochastic matrices, show that AB is
also row stochastic.
9. Prove that for an irreducible Markov chain with M + 1 states, it is
possible to go from one state to another in at most M steps.
10. A virus is found to exist in N different strains and in each generation
either stays the same or mutates into another strain with probability
p/(N − 1). Show that the probability that the strain in the n-th
generation is the same as that at time zero is
1
N
+
(
1− 1
N
)(
1− pN
N − 1
)n
.
[Hint: Model this as a 2-state Markov process with the two states
being “initial strain” and “other strain”.] 