Solutions for STAT0005 Examination Page 1
STAT0005: Probability and Inference, Level 5
Health Warning: These solutions are incomplete. Many intermediate steps are
omitted and required explanations are not given. The sole purpose of these solutions
is to enable you to check your answers when you attempt the exam yourself. If there
had been any student in the exam who had handed in the work reproduced below
without further calculations or explanations, it would probably have resulted in an
investigation as to where the student got the solutions from before the exam.
Section A
A1 (a) A andB are not independent because P (A) = 2/3 = P (B) and so P (A)P (B) =
4/9 but P (A ∩B) = P ({ω2}) = 1/3.
They are conditionally independent given C because P (A|C) = P (ω2)/P (ω2) =
1, P (B|C) = P (ω2)P (ω2) = 1 and so P (A|C)P (B|C) = 1 = P (A ∩ B|C) =
P (ω2)/P (ω2) = 1.
(b) ... Thus the set of solutions can be described as
{(p1, 1− p1, 0)T ∈ R3 : p1 ∈ [0, 1]} ∪ {(0, 1− p3, p3)T ∈ R3 : p3 ∈ [0, 1]}
A2 (a) ... and hence FX(x) =
{
0 if x < 1
1− x−2 if x ≥ 1 .
(b) As pointed out in part (a), X and Y are independent due to the independence
lemma. Hence their covariance is zero.
(c) For the reasons already given (omitted in these solutions!) above, X and Y
are independent.
A3 (a) The explanation should cover the following points:
Obtain the marginal mgf
Turn Over
Solutions for STAT0005 Examination Page 2
by evaluating for one argument set to zero
check whether product of marginal mgfs equals joint mgf
The mgf needs to exist (finite in an open neighbourhood of zero was the
standard condition used in the course)
clarity
(b) In the first case, the mgf does not exist because ...
Hence the method from part (a) will not work regardless of the distribution
of Y .
For the second case, the mgf exists (no need to compute) and hence the
method could in principle be applied
A4 (a) The likelihood is the joint pdf evaluated at the data viewed as a function of
the parameters. Since we’re dealing with a random sample, the joint pdf is
given as a product of the marginal pdfs.
. . .
⇒ `(%) = n log %− (%+ 1)
n∑
i=1
log(xi)
(b) We differentiate
solve for a critical point
Compute second derivative
check the sign of the second derivative
and check for a boundary maximum
∂`
∂%
=
n
%
−
∑
log xi
%̂ =
(
1
n
n∑
i=1
log xi
)−1
∂2`
∂%2
= − n
%2
< 0
lim
%→∞
`(%) = −∞
lim
%↓0
`(%) = −∞
The limit for %→∞ should be justified by saying that...
Continued
Solutions for STAT0005 Examination Page 3
Section B
B1 (a) The log likelihood is `(µ) = − 1
µ
∑n
i=1 ixi − n log µ (up to constants wrt µ).
The MLE is found by taking the derivative ∂`
∂µ
(µ) and equating to zero yielding
µ̂MLE =
1
n
n∑
i=1
ixi
The expectation ... so that it has zero bias.
The variance calculation crucially uses ... and results in Var(µ̂MLE) =
µ2
n
and
in the absence of bias this equals the mse.
(b) The distribution of the sample minimum results from a calculation that was
practiced in a worked example during the lecture for i.i.d. exponential case.
Following the same reasoning applied in the lecture to obtain the distribution
of the sample mimimum, we get to
F (1)(x) = . . . = 1− exp
(
−n(n+ 1)
2µ
x
)
where ... Hence, X(1) ∼ Exp(n(n+1)
2µ
) and so (or by differentiation of the cdf),
we have the pdf
f (1)(x) =
n(n+ 1)
2µ
exp
(
−n(n+ 1)
2µ
x
)
Based on the pdf, the MLE given the sample minimum is obtained in an
analogous fashion through
`(µ) = − log µ− n(n+ 1)
2µ
x(1)
∂`
∂µ
= − 1
µ
+
n(n+ 1)
2µ2
x(1)
⇒ µ̂MIN = n(n+ 1)
2
x(1)
and from what we know about the distribution ofX(1), we can infer E[µ̂MIN ] =
µ and Var(µ̂MIN) = µ
2. In the absence of bias, the mse is equal to variance
and hence µ2.
Turn Over
Solutions for STAT0005 Examination Page 4
(c) (i) Since ..., we get µ̂MIN =
n(n+1)
2
mn.
(ii) Since ..., we may infer that the other colleague’s remark is correct.
The estimator µ̂MIB = nz
(1) based on bn has mean µ and variance µ
2
whence its MSE is also µ2.
(iii) ... µ̂MIB = nbn.
Hence ...
a1 = x1, an+1 =
1
n+ 1
(nan + (n+ 1)xn+1)
(One could use other normalizations, e.g. T = nS would also work but
might pose computer problems with overflow that students are not ex-
pected to know about). The resulting estimator is µ̂MLE = an.
(d) The mse for the estimator based on mn has already been calculated in part
(b), it is µ2.
The mse for the estimator based on an has already been calculated in part
(a), it is µ2/n.
The mse for the estimator based on bn has already been calculated in part
(c)(ii), it is µ2.
Hence, the method based on an yields the smallest mean square error.
B2 (a)
E[‖X‖22] = . . . = 1 + σ21 + σ22
(b) The log likelihood is given as
`(α) = −1
2
(
x−
(
cosα
sinα
))T (
x−
(
cosα
sinα
))
+ const
∂`
∂α
=
( − sinα
cosα
)T
x
Hence, α must be such that
( − sinα
cosα
)
is orthogonal to x, or, in other words
Continued
Solutions for STAT0005 Examination Page 5
(
cosα
sinα
)
must be parallel with x. Hence, the MLE can be written as
α̂ = arctan
x2
x1
(c) ... And so the expected value is
i(α) = E
[
− ∂
2`
∂α2
]
=
( − sinα
cosα
)
Σ−1
( − sinα
cosα
)
(d)
i(α) = λ−11
(( − sinα
cosα
)T
u
)2
+ λ−12
(( − sinα
cosα
)T
v
)2
Since λ1 > λ2, this is maximized if α is such that
( − sinα
cosα
)
and u are
orthogonal.
The sketch should show an ellipse (like the one on the lecture slides when
visualizing the effect of parameter on the bivariate normal)
When i(α) is maximal, the major axis of the ellipse should be in line with
the mean vector. When i(α) is minimal, the major axis of the ellipse should
be orthogonal to the mean vector.
(e) The explanation should say that the variance of α̂ will be minimal when i(α)
is maximal
and that this happens when most of the variation is in line with the mean
vector rather than orthogonal to it (or explanation to that effect)
End of Paper