MATH6174: Likelihood and Bayesian Inference
Assignment 4
• This assignment is worth 10% of the overall mark for MATH6174.
• The deadline for submission is 16.00 on Friday 17 December.
• Standard University policies and procedures will be followed for late submission,
extensions and academic integrity.
• Please scan to a pdf any handwritten solutions. Details of how to do this with a
mobile phone are available in the Assignments folder. Additionally, please provide
the R code in a .R file.
• Submission is via Blackboard in a file called Assignment4-ID.pdf and Assignment4-ID.R,
where ID is your student ID number, for example Assignment4-12345678.pdf and
Assignment4-12345678.R. In the Assignments folder, click on Assignment 4 to sub-
mit.
Assume the following model for n random variables X1, . . . , Xn:
Xi ∼ Poisson(λ1) for i = 1, . . . , k
Xi ∼ Poisson(λ2) for i = k + 1, . . . , n,
with a conjugate prior distribution for λj being Gamma(αj, βj) with density
pi(λj) =
β
αj
j
Γ(αj)
λ
αj−1
j exp(−βjλj).
Also assume a discrete uniform prior for k on the set {1, . . . , n− 1}.
(i) Show that the joint distribution of X1 . . . , Xn, λ1, λ2 and k is
f(x1, . . . , xn, λ1, λ2, k) =
(
k∏
i=1
exp(−λ1)λxi1
xi!
)
·
(
n∏
i=k+1
exp(−λ2)λxi2
xi!
)
· β
α1
1
Γ(α1)
λα1−11 exp(−β1λ1) ·
βα22
Γ(α2)
λα2−12 exp(−β2λ2) ·
1
n− 1 .
(ii) Show that
λ1|X1, . . . , Xn, k ∼ Gamma(α1 +
k∑
i=1
xi, β1 + k)
and find the conditional distribution for λ2|X1, . . . , Xn, k.
1
(iii) Show that the conditional distribution of k given the observations, λ1 and λ2 is
proportional to
pi(k|X1, . . . , Xn, λ1, λ2) ∝ λ
∑k
i=1 xi
1 · λ
∑n
i=k+1 xi
2 · exp((λ2 − λ1) · k).
Find the normalising constant and transform this expression into a proper probability
mass function.
(iv) Write down what the Gibbs sampler step is for some t = 1, 2, . . . to sample from full
conditional distributions of λ1, λ2 and k.
(v) The following data (x1, . . . , xn) was observed (written in an R format for your con-
venience)
data <- c(4,5,4,1,0,4,3,4,0,6,3,3,4,0,2,6,3,3,5,4,5,3,1,4,4,1,5,5,3,4,
2,5,2,2,3,4,2,1,3,2,2,1,1,1,1,3,0,0,1,0,1,1,0,0,3,1,0,3,2,2,0,1,1,1,0,
1,0,1,0,0,0,2,1,0,0,0,1,1,0,2,3,3,1,1,2,1,1,1,1,2,4,2,0,0,0,1,4,0,0,0,
1,0,0,0,0,0,1,0,0,1,0,1).
Assuming the following parameters α1 = 1, β1 = 2, α2 = 2, β2 = 4, write the code
in R to implement Gibbs sampler for this case (consider t = 1, 2, . . . , 1000). (Hint:
to sample from a discrete distribution in R use the sample function and specify the
prob argument to provide the p.m.f. values).
(vi) Plot the resulting chains. From the resulting chains, calculate and plot the evolution
of the estimate of P (λ1 ≥ 3 · λ2) over time t = 1, 2, . . . , 1000. Run the sampler 100
times and plot the replications of the estimate of P (λ1 ≥ 3 ·λ2) over time in grey (all
in one plot). Add a single realisation of the evolution of the estimate of P (λ1 ≥ 3 ·λ2)
over time as a black line over the plotted range.