Final exam practice questions
for ECON3203-ECON5403
Question 1
Let x = (x1, x2, ..., xn) be an iid sample from the distribution with density
f(x|θ) = x
4
24θ5 exp(−
x
θ
), x > 0,
where θ > 0 is the parameter of the distribution. Given that the sample mean x¯ = 1
n
∑n
i=1 xi =
10, find the MLE of θ.
Solution: The likelihood function is
L(θ) = p(x|θ) =
n∏
i=1
f(xi|θ)
=
n∏
i=1
x4i
24θ5 exp(−
xi
θ
)
The log-likelihood function
ℓ(θ)=log(L(θ))=−5nlog(θ)−n
θ
x¯+C
where C is the term independent of θ. Solving for
∂ℓ(θ)
∂θ
=−5n
θ
+−5x¯
θ2
=0
we have the MLE
θ̂= x¯5 =2.
Question 2
Let x1, ..., xn be iid sample from a Negative Binomial Distribution
p(x) = Cxr+x−1θr(1− θ)x, x = 0, 1, ...
where r>0 is an integer number, θ∈(0,1) and
Cxr+x−1 =
(r + x− 1)!
x!(r − 1)!
is the binomial coefficient. Suppose that r=5 and x¯=5, find the MLE of θ.
Answer
L(θ|x1, .., xn) =
n∏
i=1
p(xi) =
n∏
i=1
Cxir+xi−1θ
r(1− θ)xi = θnr(1− θ)
∑n
i=1 xi
n∏
i=1
Cxir+xi−1
The log-likelihood
ℓ(θ|x1, ..., xn) = logL(θ|x1, ..., xn) = nr log(θ) + log(1− θ)
n∑
i=1
xi + log
n∏
i=1
Cxir+xi−1
= nr log(θ) + nx¯ log(1− θ) + log
n∏
i=1
Cxir+xi−1
The score function
∂ℓ(θ|x1, ..., xn)
∂θ
= nr
θ
− nx¯1− θ ,
whose solution is
θ̂ = r
r + x¯ =
5
5 + 5 = 0.5.
Question 3
Present the Maximum Likelihood Estimation method: explain what it is, how to use it.
Question 4
The spam email data set created by Mark Hopkins, Erik Reeber, George Forman and Jaap
Suermondt at the Hewlett-Packard Labs consists of 4061 messages, each has been already
classified as proper email or spam together with 57 attributes (predictors) which are relative
frequencies of commonly occurring words. The goal is to design a spam filter that could filter
out spam. Which statistical model that you have learnt can be applied in this applications?
Denote the data set as {yi,xi,i=1,...,n}. Write down the log-likelihood function based on the
model that you selected.
Solution: This is a regression problem with a bianry response variable, hence a logistic
regression model can be used. See the lecture notes for the log-likelihood function of a
logistic regression model.
Question 5
Percentage of body fat is one important measure of health, which can be accurately estimated
by underwater weighing techniques. These techniques often require special equipment and
are sometimes not convenient, thus fitting percent body fat to simple body measurements
is a convenient way to predict body fat. Johnson (1996) introduced a dataset in which
percent body fat and 13 simple body measurements (such as weight, height and abdomen
circumference) are recorded for 252 men. Which statistical model that you have learnt can
be applied in this applications? Denote that data set as {yi,xi,i=1,...,n}. Write down the
likelihood function based on the model that you selected.
Page 2
Solution: This is a regression problem with a continuous response variable, hence a linear
regression model can be used. See the lecture notes for the log-likelihood function of a
linear regression model.
Question 6
The manager of the purchasing department of a large company would like to develop a re-
gression model to predict the average amount of time it takes to process a given number of
invoices. The following model was fit to the data: Y =β0+β1x+e where Y is the processing
time and x is the number of invoices. Utilizing the output from the fit of this model provided
below, complete the following tasks.
1. Find a 95% confidence interval for the start-up time, i.e. β0.
Given a large data size n, the critical value tn−2,0.025≈z0.025=1.96.
2. Suppose that a best practice benchmark for the average processing time for an additional
invoice is 0.01 hours (or 0.6 minutes). Test the null hypothess H0 :β1=0.01 against a
two-sided alternative. Calcuate the tstat for this test
. reg Time Invoices
Source | SS df MS Number of obs = 30
-------------+------------------------------ F( 1, 28) = 190.36
Model | 20.7019874 1 20.7019874 Prob > F = 0.0000
Residual | 3.04501264 28 .108750451 R-squared = 0.8718
-------------+------------------------------ Adj R-squared = 0.8672
Total | 23.747 29 .818862069 Root MSE = .32977
------------------------------------------------------------------------------
Time | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
Invoices | .0112916 .0008184 13.80 0.000 ------- ---------
_cons | .6417099 .1222707 5.25 0.000 ------- --------
------------------------------------------------------------------------------
Solution: Easy; students to do
Question 7
Suppose that (x1,y1),...,(xn,yn) are n observations. Consider the simple linear regression
model,
yi=β0+β1(xi−x)+ei ei∼N(0,σ2) (1)
with the ei independent. Let βˆ0 and βˆ1 be the least squares estimates of β0 and β1. Let ϵˆi be
the ith residual, i.e. ϵˆi=yi−βˆ0−βˆ1(xi−x).
Page 3
1. Please choose the correct answer:
(a)
n∑
i=1
(xi−x)=3.
(b)
n∑
i=1
(xi−x)=−1.
(c)
n∑
i=1
(xi−x)=0.
(d) None of the above
Solution: Correct answer is (C) as
n∑
i=1
(xi−x)=
n∑
i=1
xi−nx)=0.
2. Let ê be the mean of the residuals. Please choose the correct answer:
(a) ê=n.
(b) ê=0
(c) ê=−n.
(d) None of the above
Solution: Correct answer is (B).
3. Please choose the correct answer from (a)–(d) for the least squares estimate of β̂0 of β0.
(a) 0
(b) y, the sample mean of the yi.
(c) y1+y2+y3.
(d) None of the above.
Solution: Correct answer is (B).
Question 8
We want to fit a ridge regression model to a dataset of n= 97 observations. To find the
Page 4
optimal shrinkage, we create a range of 10 values for λ :0,0.1,0.2,...,0.9, and compute the ridge
estimates β̂ridgeλ at each of these values. Let σ̂2λ=
∥y−Xβ̂ridge
λ
∥2
n
be the estimate of the variance
of the error term ε. The table below gives the values of σ̂2λ:
λ 0 0.1 0.2 0.3 0.4 0.5 .6 .7 .8 .9
σ̂2λ 0.9033 1.0812 0.8865 4.1080 4.6396 5.1962 5.7991 6.4485 7.1444 7.4611
The eigenvalues of X ′X are 1.2, 0.9, .5, .3, .2, .05 and 0.01. Find the best value of λ among
these 10 values using the AIC criterion.
Solution: AIC values are 4.1350, 16.5640, -4.2115, 143.5551, 154.6495, 165.0889, 175.2929,
185.2206, 194.8497, 198.7892. Hence, optimal λ=0.2.
Question 9
Consider the previous question, find the best value of λ among these 10 values using the BIC
criterion.
Question 10
(a) You are about to fit a dataset to the following neural network. What is the number of
Figure 1: Neural network.
layers? What is the number of hidden layers? What is the number of input neurons?
What is the number of hidden neurons? What is the number of output neurons?
(b) Describe your strategy to prepare training data for training this neural network.
Solution: Students to do
Question 11
Given a trained neural network as the figure below. Suppose you are using this network for
Page 5
Figure 2: Trained Neural network.
regression, with the ReLu activation function on all the hidden neurons; b1,b2,b3 are the bias
terms. Given an input x1=−1,x2=2,x3=−3, what is your prediction of the output y given
this network?
Solution: The first hidden unit is
z1=Relu(b1+0.1x1+0.3x2+0.5x3)=Relu(0)=0
The second hidden unit is
z2=Relu(b2+0.2x1+0.4x2+0.6x3)=Relu(0.8)=0.8
As this network is used for regression, the identity activation function is used for the
output unit, the prediction of y is
η=b3+0.7z1+0.8z2=0.64.