xuebaunion@vip.163.com

3551 Trousdale Rkwy, University Park, Los Angeles, CA

留学生论文指导和课程辅导

无忧GPA：https://www.essaygpa.com

工作时间：全年无休-早上8点到凌晨3点

微信客服：xiaoxionga100

微信客服：ITCS521

程序代写案例-STAT4207/5207-assignment 1

时间：2021-01-26

STAT4207/5207 Homework assignment 1

Due by February 10, 2021. Make sure you justify your answer. Write down the complete

derivations for full credits. Note that questions in which you are asked to classify the states, you

must specify whether they are transient, null recurrent, positive recurrent and you must specify their

period. These concepts will be discussed in lecture 3-5.

Exercise 1

Suppose that whether it rains or not depends on previous weather conditions through the last three

days. Suppose that if it has rained for the past three days, then it will rain today with probability

0.7; if it did not rain for any of the past three days, then it will rain with probability 0.3; and in any

other case the weather today will, with probability 0.6, be the same as the weather yesterday.

a) Show how this system may be analyzed by defining an appropriate Markov chain. How many

states are needed?

b) Determine the one step transition probability matrix P for this Markov chain.

Exercise 2

Exercise 14 of chapter 4 of your textbook (11th or 12th edition).

Exercise 3

Suppose P ∈ Rk×k and Q ∈ Rk×k are stochastic matrices. Show that P · Q is also a stochastic

matrix.

Exercise 4

Suppose you have two urns with a total of 5 balls. At each step, one of the five balls is chosen at

random and switched from its urn to the other urn. Let Xn be the number of balls in the first urn

after n switches.

a) Is {Xn : n = 1, 2, . . .} a Markov Chain? Explain.

b) Define the state space and provide the one step transition matrix.

c) Draw the corresponding transition graph.

d) Classify the states. Are there any periodic states?

e) What is the probability that, given I have 3 balls in the first urn at the 10th turn, I will have 2

balls at the 12th turn?

Exercise 5

Consider the pessimistic covid-19 model from Lecture 3 with state space S = {0, 1, 2, 3} and transi-

tion matrix

P =

0.6 0.4 0.0 0

0.3 0.5 0.2 0

0 0.4 0.5 0.1

0 0 0 1

1

a) Draw the transition graph.

b) Classify the states. Are there any periodic states?

c) Does this chain have a steady state distribution? Explain why or why not. If you answered yes,

derive the steady state distribution.

d) Suppose P(X0 = 0) = 0.7,P(X0 = 1) = 0.1,P(X0 = 2) = 0.05. Find E[X2].

Exercise 6

Consider a Markov Chain with state space ={0, 1, 2, 3}

P =

0.3 0.7 0 0

0.2 0.5 0.3 0

0 0.3 0.4 0.3

0 0 0.6 0.4

a) Draw the transition graph.

b) Does this chain have a steady state distribution? Explain why or why not. If you answered yes,

derive the steady state distribution.

c) What is the expected time to visit state 1 for the first time, given the chain starts in state 0?

d) Find limn→∞ P(Xn+3 = 1, Xn+5 = 3|X0 = 2). Do not forget to justify your answer.

e) Let A = {0, 3}. What is the probability that the chain reaches state 2 before it reaches any state

in A, given the chain starts in state 1?

2

Due by February 10, 2021. Make sure you justify your answer. Write down the complete

derivations for full credits. Note that questions in which you are asked to classify the states, you

must specify whether they are transient, null recurrent, positive recurrent and you must specify their

period. These concepts will be discussed in lecture 3-5.

Exercise 1

Suppose that whether it rains or not depends on previous weather conditions through the last three

days. Suppose that if it has rained for the past three days, then it will rain today with probability

0.7; if it did not rain for any of the past three days, then it will rain with probability 0.3; and in any

other case the weather today will, with probability 0.6, be the same as the weather yesterday.

a) Show how this system may be analyzed by defining an appropriate Markov chain. How many

states are needed?

b) Determine the one step transition probability matrix P for this Markov chain.

Exercise 2

Exercise 14 of chapter 4 of your textbook (11th or 12th edition).

Exercise 3

Suppose P ∈ Rk×k and Q ∈ Rk×k are stochastic matrices. Show that P · Q is also a stochastic

matrix.

Exercise 4

Suppose you have two urns with a total of 5 balls. At each step, one of the five balls is chosen at

random and switched from its urn to the other urn. Let Xn be the number of balls in the first urn

after n switches.

a) Is {Xn : n = 1, 2, . . .} a Markov Chain? Explain.

b) Define the state space and provide the one step transition matrix.

c) Draw the corresponding transition graph.

d) Classify the states. Are there any periodic states?

e) What is the probability that, given I have 3 balls in the first urn at the 10th turn, I will have 2

balls at the 12th turn?

Exercise 5

Consider the pessimistic covid-19 model from Lecture 3 with state space S = {0, 1, 2, 3} and transi-

tion matrix

P =

0.6 0.4 0.0 0

0.3 0.5 0.2 0

0 0.4 0.5 0.1

0 0 0 1

1

a) Draw the transition graph.

b) Classify the states. Are there any periodic states?

c) Does this chain have a steady state distribution? Explain why or why not. If you answered yes,

derive the steady state distribution.

d) Suppose P(X0 = 0) = 0.7,P(X0 = 1) = 0.1,P(X0 = 2) = 0.05. Find E[X2].

Exercise 6

Consider a Markov Chain with state space ={0, 1, 2, 3}

P =

0.3 0.7 0 0

0.2 0.5 0.3 0

0 0.3 0.4 0.3

0 0 0.6 0.4

a) Draw the transition graph.

b) Does this chain have a steady state distribution? Explain why or why not. If you answered yes,

derive the steady state distribution.

c) What is the expected time to visit state 1 for the first time, given the chain starts in state 0?

d) Find limn→∞ P(Xn+3 = 1, Xn+5 = 3|X0 = 2). Do not forget to justify your answer.

e) Let A = {0, 3}. What is the probability that the chain reaches state 2 before it reaches any state

in A, given the chain starts in state 1?

2