THE UNIVERSITY OF NEW SOUTH WALES
DEPARTMENT OF STATISTICS
PRACTICE MIDTERM TEST - 2023 , Week 6
SOLUTIONS
MATH5905
Time allowed: 135 minutes
1. Let X = (X1, X2, . . . , Xn) be i.i.d. Poisson(θ) random variables with density function
f(x, θ) =
e−θθx
x!
, x = 0, 1, 2, . . . , and θ > 0.
a) The statistic T (X) =
∑n
i=1Xi is complete and sufficient for θ. Provide justifi-
cation for why this statement is true.
Using the property of the one-parameter exponential family, we ob-
serve that
f(x, θ) =
e−θθx
x!
= e−θ
1
x!
exp
(
x log θ
)
Thus, the Poisson distribution belongs to the one-parameter expo-
nential family. Then this implies that T (X) =
∑n
i=1Xi is complete and
(minimal) sufficient for θ.
b) Derive the UMVUE of h(θ) = e−kθ where k = 1, 2, . . . , n is a known integer.
You must justify each step in your answer. Hint: Use the interpretation that
P (X1 = 0) = e
−θ and therefore P (X1 = 0, . . . , Xk = 0) = P (X1 = 0)k = e−kθ.
Since T is sufficient and complete for θ, we first need to find an unbi-
ased estimator of h. Let W = I{X1=0,...,Xk=0}(X) and then using the fact
that P (Xi = 0) = e
−θ we see that
E(W ) = P (X1 = 0, . . . , Xk = 0) = P (X1 = 0)
k =
[
e−θ
]k
= e−kθ,
which is unbiased for h(θ). Now, we apply the Theorem of Lehmann-
Scheffe, and obtain the following
1
τˆ(T ) = E(W |T = t) = E
(
I{X1=0,...,Xk=0}|
n∑
i=1
Xi = t
)
= P
(
X1 = 0, . . . , Xk = 0|
n∑
i=1
Xi = t
)
=
P
(
X1 = 0, . . . , Xk = 0,
∑n
i=1Xi = t
)
P
(∑n
i=1Xi = t
)
Now we observe that the events in the numerator must be satisfied
simultaneously to have a non-zero probability and hence this reduces
to
τˆ(T ) =
P
(
X1 = 0, . . . , Xk = 0,
∑n
i=k+1Xi = t
)
P
(∑n
i=1Xi = t
)
= e−kθ
P
(∑n
i=k+1Xi = t
)
P
(∑n
i=1Xi = t
)
= e−kθ
e−θ(n−k)(θ(n− k))t
t!
· t!
e−nθ(nθ)t
=
(
n− k
n
)t
=
(
1− k
n
)nX¯
which is the UMVUE for h(θ) = e−kθ.
c) Calculate the Cramer-Rao lower bound for the minimal variance of an unbiased
estimator of h(θ) = e−kθ.
First we calculate the fisher information as follows. We have that
log f(x) = −θ + x log θ − log x!
∂
∂θ
log f(x) = −1 + x
θ
∂2
∂θ2
log f(x) = − x
θ2
and so the Fisher information in a single sample is
2
IX1(θ) = −E
[
∂2
∂θ2
log f(x)
]
=
E(X)
θ2
=
θ
θ2
=
1
θ
Hence the Fisher information for the whole sample is
IX = nIX1(θ) =
n
θ
.
Then notice that
∂
∂θ
h(θ) = −ke−kθ.
Hence the Cramer-Rao lower bound is(
∂
∂θ
h(θ)
)2
IX(θ)
=
θ
n
k2e−2kθ
d) Show that there does not exist an integer k for which the variance of the UMVUE
of h(θ) attains this bound.
To show that the Cramer-Rao bound is attainable we will look at the
score function:
V (X, θ) = −n+ 1
θ
n∑
i=1
Xi
= −n+ nX¯
θ
=
n
e−kθ
(
X¯e−kθ
θ
− e−kθ
)
Therefore, no matter what integer k we choose the term X¯e
−kθ
θ
cannot
be a statistic as it will still depend on θ.
e) Determine the MLE hˆ of h(θ).
The MLE for θ is simply θˆ = X¯ and hence the
ĥ(θ)mle = h(θˆmle) = e
−kX¯ .
f) Suppose that n = 5, T = 10 and k = 1 compute the numerical values of the
UMVUE in part (b) and the MLE in part (e). Comment on these values.
The UMVUE is
ĥ(θ)umvue =
(
1− 1
5
)5×2
= 0.107
The MLE is
ĥ(θ)mle = e
−2 = 0.135
As n → ∞ the UMVUE would converge to the MLE while for finite
sample size these values are slightly different.
3
g) Consider testing H0 : θ ≤ 2 versus H1 : θ > 2 with a 0-1 loss in Bayesian setting
with the prior τ(θ) = 4θ2e−2θ. What is your decision when n = 5 and T = 10.
You may use: ∫ 2
0
x12e−7xdx = 0.00317
Note: The continuous random variable X has a gamma density f with param-
eters α > 0 and β > 0 if
f(x;α, β) =
1
Γ(α)βα
xα−1e−x/β
and
Γ(α + 1) = αΓ(α) = α!
First we need to compute the posterior by observing that it propor-
tional to the likelihood times the prior. The likelihood is
L(X|θ) = e
−nθθ
∑n
i=1Xi∏n
i=1Xi!
∝ e−5θθ10
therefore
h(θ|x) ∝ 4θ2e−2θ × e−5θθ10 = θ12e−7θ
which implies that
θ|X ∼ gamma(13, 1
7
).
Hence we are interested in computing the posterior probability
P (θ < 2|X) =
∫ 2
0
1
Γ(13)
(
1
7
)13 θ12e−7θdθ
= 202.27× 0.00317
= 0.64
We compare this posterior probability with 0.5 since we are dealing
with a 0-1 loss. Since this probability is greater than 0.5 we must
NOT reject H0.
4
2. Let X1, X2, . . . , Xn be independent random variables, with a density
f(x; θ) =
{
e−(x−θ), x > θ,
0 else
where θ ∈ R1 is an unknown parameter. Let T = min{X1, . . . , Xn} = X(1) be the
minimal of the n observations.
a) Show that T is a sufficient statistic for the parameter θ.
First calculate the likelihood as follows
L(X, θ) =
n∏
i=1
e−(Xi−θ)I(−∞,Xi)(θ) = exp
(
−
n∑
i=1
Xi + nθ
)
I(−∞,X(1))(θ)
Therefore the likelihood can be written as L(X, θ) = g(T, θ)h(X) where
g(T, θ) = exp(nθ)I(−∞,T )(θ) and h(X) = exp
(
−
n∑
i=1
Xi
)
Hence, T = X(1) is sufficient by the Neyman Fisher Factorization Cri-
terion.
b) Show that the density of T is
fT (t) =
{
ne−n(t−θ), t > θ,
0 else
Hint: You may find the CDF first by using
P (X(1) < x) = 1− P (X1 > x ∩X2 > x · · · ∩Xn > x).
First note that for x < θ we have P (X1 ≥ x) = 1 and for x ≥ θ we have,
P (X1 ≥ x) =
∫ ∞
x
e−(y−θ)dy =
[
− e−(y−θ)
]y=∞
y=x
= e−(x−θ)
Hence,
FT (t, θ) = P (T ≤ t)
= 1− P (X1 ≥ t,X2 ≥ t, . . . , Xn ≥ t)
= 1− P (X1 ≥ t)n
=
{
1− e−n(t−θ) if t ≥ θ
0 if t < θ.
Then by differentiation
fT (t, θ) = ne
−n(t−θ), t ≥ θ,
otherwise zero.
5
c) Find the maximum likelihood estimator of θ and provide justification.
The MLE for θ is calculated by maximizing over all θ values
L(X, θ) = exp
(
−
n∑
i=1
Xi + nθ
)
I(−∞,X(1))(θ)
The graph of the Likelihood function is at zero at minus infinity and
then continues to increase until the value X(1) and then from here
stays at zero. Hence X(1) must be the MLE.
d) Show that the MLE is a biased estimator. Hint: You might want to consider
the transformation Y = T − θ when performing the integral and then utilize the
density of an exponential distribution.
By calculating
E(X(1)) =
∫ ∞
θ
t · ne−n(t−θ)dt
Now let y = t− θ then we get
E(X(1)) = n
∫ ∞
0
(y + θ)e−nydy
=
∫ ∞
0
y · ne−nydy + θ
∫ ∞
0
ne−nydy
=
1
n
+ θ and hence the MLE is a biased estimator.
e) Show that T = X(1) is complete for θ.
Take any function g such that Eθg(T ) = 0 ∀θ. Then
0 = Eθg(T ) =
∫ ∞
−∞
g(t)fT (t)dt
=
∫ ∞
θ
g(t) · ne−n(t−θ)dt
= nenθ
∫ ∞
θ
g(t)e−ntdt
This implies that ∫ ∞
θ
g(t)e−ntdt = 0 ∀θ
By taking the derivative with respect to θ to both sides we get
−g(θ)e−nθ = 0 ∀ θ
but since e−nθ ̸= 0 we must have g(θ) = 0 for all θ. Since θ is just acting
as a variable we have g(t) = 0 for all t. Hence P (g(T ) = 0) = 1 for all θ
and hence T = X(1) is complete for θ.
6
f) Hence determine the UMVUE of θ.
From part d) we know that W = X(1) − 1n is an unbiased estimator for
θ. Thus by Lehmann-Scheffe theorem,
θˆ = E(W |X(1)) = X(1) − 1
n
is the unique UMVUE of θ.