Homework 4
MAS 442 Spring 2020
Due: 11:59 PM Apr 6
Problem 1: One of the most widely used policies to manage inventory subject to random
demand is the “(s, S)” policy. When the inventory level at the end of the day falls at s or
below, we order enough inventory to bring it back at S. For simplicity, we assume that there
is no lead time.
Suppose now that an electronics store sells a video game console and uses an inventory
policy with s = 2 and S = 6. That is, if the inventory level at the end of the day is either 0 or
1 or 2, then they order enough new units so that the total inventory level at the beginning of
the next day is 6. Let Xn denote the inventory level at the end of day n. The daily demands
are independent random variables following the Binomial (4, 0.6) distribution. Today’s (day
0) end-of-day inventory is 3.
(a) Model the inventory level at the end of each day (day n) as a Markov Chain. Clearly
define S, α(0), P . Also present the p.m.f of the demand. (20 Points)
(b) Compute the expected inventory level at the end of the day 5. (5 Points)
(c) Compute the probability of a stock-out during day 6. (5 Points)
1
Problem 2: (Ehrenfest model) Gas molecules move about randomly in a box which is
divided into two halves symmetrically by a partition. A hole is made in the partition.
Suppose there are 20 molecules in the box. Think of the partitions as two urns (urn 0 and
1) containing balls labeled 1 through 20. Molecular motion can be modeled by choosing a
number between 1 and 20 at random and moving the corresponding ball from the urn it
is presently into the other. This is a historically important physical model introduced by
Ehrenfest in the early days of statistical mechanics to study thermodynamic equilibrium. The
Ehrenfest model has also been applied to study the dynamics of social network formations.
Let Xn denote the number of molecules at the left partition of the box (urn 0) after n
transitions.
(a) Draw a transition diagram of the process. You can draw this for the first three and last
three states, and leave . . . in between. Is {Xn : n ≥ 0} irreducible? (10 Points)
(b) Argue that {Xn : n ≥ 0} is a DTMC. In particular, argue that Xn is a finite-space
random walk. Clearly define S, and P . For P , complete the following matrix. (20
Points)
P =

? ? ? ? . . . ? ? ?
? ? ? ? . . . ? ? ?
? ? ? ? . . . ? ? ?
...
...
...
...
. . .
...
...
...
? ? ? ? . . . ? ? ?
? ? ? ? . . . ? ? ?

.
(c) Write a system of equations for the stationary distribution (not in matrix form). You
can only write the first three and the last three equations and leave
... between them.
Solve the system in MATLAB. Copy your MATLAB code, and attach the histogram of
the steady-state probability. (20 Points)
2
Problem 3: Consider the following simple model for the random evolution of a stock
price for a certain stock (lets call it stock A). Each day, the stock price increases by
$1 with probability r = 0.17, moves down $1 with probability p = 0.15, or remains the
same with probability q = 1 − p − r. Today (day zero), the stock price is $13. Stock
price of $0 is an absorbing state; once the stock price hits zero, it remains there forever
(out of business etc.). Consider the following trading strategy: we hold on to the stock
until its price becomes $35, at which point we sell.
(a) Draw a transition diagram of the process. You can draw this for the first three and
last three states, and leave . . . in between. Is this stochastic process irreducible?
Explain your answer. (10 Points)
(b) Model the random evolution of the stock price as a DTMC with two absorbing states.
Clearly define S, α(0), P . For P , complete the following matrix.(15 Points)
P =

? ? ? ? . . . ? ? ?
? ? ? ? . . . ? ? ?
? ? ? ? . . . ? ? ?
...
...
...
...
. . .
...
...
...
? ? ? ? . . . ? ? ?
? ? ? ? . . . ? ? ?

.
(c) What is the expected stock price at the end of 10th day? (5 Points)
3























































































学霸联盟