ST407 Monte Carlo Methods 2021/22
Assignment 2
Handed out: Monday, November 22nd, 2021
Deadline: 1:00 PM on Thursday, 2 December 2021
1. (a) Implement a simple random walk Metropolis algorithm to target the generalised Gaussian dis-
tribution, with density:
f(x;µ, σ, β) ∝ exp
(
−
∣∣∣∣x− µσ
∣∣∣∣β
)
of location µ ∈ R, scale σ ∈ (0,∞) and shape β ∈ [1, 2]; you may assume that µ = 0 and σ = 1
throughout this question.
Run this algorithm to produce a sample with which the f(x; 0, 1, 1.5) can be approximated.
Explain:
i. How you chose the (scale of) the proposal distribution.
ii. How you decided how long a burn-in period (if any) was required.
iii. How you assessed whether the chain had converged adequately in the number of iterations
used.
(b) Estimate the second moment of f with µ = 0, σ2 = 1 and β = 1.5 using the standard estimator.
Repeat this process several times to assess the variance of your estimator.
(c) Explain briefly1 how you might go about estimating the variance of f(x; 0, 1, β) (i.e. the variance
of a random variable distributed according to f(·; 0, 1, β))for a range of values of β using a single
Markov chain. Explain in particular what the invariant distribution of this Markov chain would
be, why this would be a good choice and how you would compute the variance of Xβ with
distribution f(x; 0, 1, β) for any β ∈ [1, 2] using the resulting ouput.
2. Consider the following algorithm to sample from a distribution with density f(x). Starting with an
initial value X(0) ∈ supp(f), iterate for t = 1, 2, . . ..
1. Draw U (t) ∼ U[0, f(X(t−1))].
2. Draw X(t) ∼ U{x : f(x) ≥ U (t)}.
where U[a, b] denotes the uniform distribution on interval [a, b].
The figure below illustrates the above algorithm:
1A paragraph or two should suffice.
0.0 0.2 0.4 0.6 0.8 1.0
0 .
0
0 .
5
1 .
0
1 .
5
2 .
0
x
u
slicebeta
This type of data augmentation (adding the variable U to the state of interest) is known as slice
sampling.
(a) Show that the above algorithm is a Gibbs sampler that samples from the uniform distribution
on the set {(x, u) : u ≤ f(x)}.
(b) Deduce from this that the invariant distribution of X(t) has the density f(·).
(c) For the density
f(x) =
3
8
[
I[−5,−3](x)(1− (x+ 4)2) + I[+3,+5](x)(1− (x− 4)2)
]
where
I[a,b](x) =
{
1 if x ∈ [a, b]
0 otherwise
i. What would the two steps of a slice sampler iteration be? State each step explicitly for this
particular distribution.
ii. Implement such a slice sampler and a random walk Metropolis algorithm to sample from f
and estimate the mean and variance of f using a few thousand iterations of each. You might
wish to consider several proposal scales for the Metropolis algorithm. Include the source
code of your implementation in your solutions.
iii. Using simple plots, compare qualitatively the behaviour of the two algorithms.
(d) Slice sampling can be difficult to implement for real problems, particularly when the target
distribution has dimension larger than one.
i. Why? [Consider how you would implement it for a complex distribution.]
ii. Propose another MCMC algorithm with the same target distribution as the slice sampler
which would not suffer from this problem. [Hint: which step isn’t easy to implement? How
might the troublesome variable be updated without using a Gibbs sampling step?]
iii. Compare your proposed algorithm with the two considered in part c and explain what you