微信客服：xiaoxionga100
微信客服：ITCS521
无代写-ELEC 9741-Assignment 1
时间：2022-06-20
ELEC 9741: Assignment 1, 2022
Instructions
1 due in Moodle, Friday June 24, 4pm
2 Signed School Cover Sheet attached
3 TYPED PDF only - no microsoft word docs.
4 Follow the Homework Rules.
5 Computeroutput : no commentary⇒ no marks.
6 Analyticalresults : no working⇒ no marks.
7 ♦ means you can use Matlab; else not.
8 No Copyingexcept from lectures•
No Discussion
No web searching
Q1 (14) Theory
(a) If θˆ is an estimator of a vector parameter θ show that
MSE = E(‖ θˆ− θ ‖2) =‖ bias ‖2 +trace(var(θˆ))
(a) For the regression problem y = Xn×pβ + where
∼ N(0, σ2I)with true value βo, consider the biased
(’shrinkage’) estimator
βˆa = (1− a
T
)βˆ
βˆ = (X>X)−1X>y = LSE
(i) Find the bias and variance of βˆa.
(ii) Show βˆa has a smaller MSE than the LSE when
a <
2τ
vsnr + τT
where vsnr = ‖βo‖
2
σ2 and τ = T
∑p
1
1
λ2
k
and λk
are the singular values of X .
(b) Noise Model.
Consider the stationary process
Yt = a+ φYt−3 + t, t = 1, 2, · · ·
where t is a Gaussian white noise sequence of zero
mean and variance σ2.
(i) Explain what are the stability/stationarity con-
straints on φ?
(ii) Derive closed form expressions for the mean and
acs of Yt.
Q2(14) (Impulse Response Estimation)
Consider a system with output sequence measured in noise
yt = st + nt, st = (h ∗ u)t, t = 1, · · · , T
with impulse response hr = Aγrsin(2pi rTo ), r = 1, 2, · · · ,m.
The input sequence is a (0, σ2u) white noise
1 independent
of the observation noise sequence which is a (0, σ2) white
noise sequence. The variance SNR is vsnr = σ
2
s
σ2u
.
(a) Ignoring start-up transients, and setting ωo = 2piTo ,
show that for large m
σ2s =
∑m
1 h
2
rσ
2
u ≈ σ2u
∑∞
1 h
2
r
= A2σ2u
γ2
2
[
1
(1− γ2) +
cos(2ωo)− γ2
1 + γ4 − 2γ2cos(2ωo) ]
(b) ♦Simulation.
Write an m-file to simulate the system for T = 100
with the values: [γ, To, σ, vsnr] = [.8, 7, 1, 1] and
m = mo = 20.
(c) ♦Show four displays:
plots of st, yt on the same graph;
histograms of yt, st on the same graph;
What do these plots reveal about the signals?
an impulse response plot;
a Bode plot i.e. system frequency response.
(c) ♦Parameter Estimation.
Write an m-file to compute the penalized least squares
estimator, its standard errors2, the singular values of
the X matrix.
(d) To compute the standard errors you need to derive the
following formula for the variance of the smoothness
penalized least squares estimator (SP-LSE).
var(βˆλ) = σ
2(X ′X+λD>D)−1X>X(X ′X+λD>D)−1
(e) ♦Using the simulated data from (b) compute the SP-
LSE of β for m = mo and a grid of λ values as fol-
lows.
Plot the ’loss’ = ‖ βˆλ − βo ‖2 versus λ to find the
minimizing value of λ. 3
Compute the corresponding ’minimizing’ SP-LSE, its
standard errors and plot the estimated IR overlaid with
the true IR and a 95% confidence interval. Also plot
the singular values of the X matrix and comment on
them.
Q3 (8). ♦ Statistical Graphics.
The graphics/plots you display in Q1, Q2, Q3 will earn up
to 8 marks.
1i.e. it has zero mean and variance σ2u
2se(βˆr) =
√
var(βˆr), r = 1, · · · ,m
3Such a plot is only possible in a simulation, not with real data.
Q4(14) (Noise Modeling)
Do not use any specialised matlab commands such as zp2tf,
arima, aic, bic etc.
(a) ♦Write an mfile to simulate a stationary AR(3) time
series driven by a zero mean Gaussian white noise of
unit variance.
Your mfile should accept as input, three real roots or
one real root and a complex root; all non-zero.
It should produce the AR parameters & variance di-
rectly as well as the simulated values as output.
Show two simulations (T=200) (on a single page) one
for each of the above cases. List the two sets of pa-
rameters used. In each case ensure that γo ≥ 3.
(b) ♦ Using your mfile simulate an AR(3) with roots
(.9,.1,.6) for T=200. List the true parameter values.
Using least squares regression4 produce estimates for
the 3 parameters, the noise variance as well as stan-
dard errors for the parameters.
Are the estimates within 2 standard errors of the true
values?
(c) ♦ Using your mfile simulate new data (T=100) from
the same model (ii) compute BIC5 and find its mini-
mizing order p∗. Show a single plot of BIC together
with its two components.
Give the parameter estimates corresponding to p∗ and
their standard errors.
Also do a statistical model diagnosis using just the acs
of the residuals. What conclusions do you draw about
the quality of the estimated parameters and model or-
der?
4write your own mfile; don’t use any matlab command for any regres-
sion related computations
5using your own mfile; not matlab’s BIC command
Instructions
1 due in Moodle, Friday June 24, 4pm
2 Signed School Cover Sheet attached
3 TYPED PDF only - no microsoft word docs.
4 Follow the Homework Rules.
5 Computeroutput : no commentary⇒ no marks.
6 Analyticalresults : no working⇒ no marks.
7 ♦ means you can use Matlab; else not.
8 No Copyingexcept from lectures•
No Discussion
No web searching
Q1 (14) Theory
(a) If θˆ is an estimator of a vector parameter θ show that
MSE = E(‖ θˆ− θ ‖2) =‖ bias ‖2 +trace(var(θˆ))
(a) For the regression problem y = Xn×pβ + where
∼ N(0, σ2I)with true value βo, consider the biased
(’shrinkage’) estimator
βˆa = (1− a
T
)βˆ
βˆ = (X>X)−1X>y = LSE
(i) Find the bias and variance of βˆa.
(ii) Show βˆa has a smaller MSE than the LSE when
a <
2τ
vsnr + τT
where vsnr = ‖βo‖
2
σ2 and τ = T
∑p
1
1
λ2
k
and λk
are the singular values of X .
(b) Noise Model.
Consider the stationary process
Yt = a+ φYt−3 + t, t = 1, 2, · · ·
where t is a Gaussian white noise sequence of zero
mean and variance σ2.
(i) Explain what are the stability/stationarity con-
straints on φ?
(ii) Derive closed form expressions for the mean and
acs of Yt.
Q2(14) (Impulse Response Estimation)
Consider a system with output sequence measured in noise
yt = st + nt, st = (h ∗ u)t, t = 1, · · · , T
with impulse response hr = Aγrsin(2pi rTo ), r = 1, 2, · · · ,m.
The input sequence is a (0, σ2u) white noise
1 independent
of the observation noise sequence which is a (0, σ2) white
noise sequence. The variance SNR is vsnr = σ
2
s
σ2u
.
(a) Ignoring start-up transients, and setting ωo = 2piTo ,
show that for large m
σ2s =
∑m
1 h
2
rσ
2
u ≈ σ2u
∑∞
1 h
2
r
= A2σ2u
γ2
2
[
1
(1− γ2) +
cos(2ωo)− γ2
1 + γ4 − 2γ2cos(2ωo) ]
(b) ♦Simulation.
Write an m-file to simulate the system for T = 100
with the values: [γ, To, σ, vsnr] = [.8, 7, 1, 1] and
m = mo = 20.
(c) ♦Show four displays:
plots of st, yt on the same graph;
histograms of yt, st on the same graph;
What do these plots reveal about the signals?
an impulse response plot;
a Bode plot i.e. system frequency response.
(c) ♦Parameter Estimation.
Write an m-file to compute the penalized least squares
estimator, its standard errors2, the singular values of
the X matrix.
(d) To compute the standard errors you need to derive the
following formula for the variance of the smoothness
penalized least squares estimator (SP-LSE).
var(βˆλ) = σ
2(X ′X+λD>D)−1X>X(X ′X+λD>D)−1
(e) ♦Using the simulated data from (b) compute the SP-
LSE of β for m = mo and a grid of λ values as fol-
lows.
Plot the ’loss’ = ‖ βˆλ − βo ‖2 versus λ to find the
minimizing value of λ. 3
Compute the corresponding ’minimizing’ SP-LSE, its
standard errors and plot the estimated IR overlaid with
the true IR and a 95% confidence interval. Also plot
the singular values of the X matrix and comment on
them.
Q3 (8). ♦ Statistical Graphics.
The graphics/plots you display in Q1, Q2, Q3 will earn up
to 8 marks.
1i.e. it has zero mean and variance σ2u
2se(βˆr) =
√
var(βˆr), r = 1, · · · ,m
3Such a plot is only possible in a simulation, not with real data.
Q4(14) (Noise Modeling)
Do not use any specialised matlab commands such as zp2tf,
arima, aic, bic etc.
(a) ♦Write an mfile to simulate a stationary AR(3) time
series driven by a zero mean Gaussian white noise of
unit variance.
Your mfile should accept as input, three real roots or
one real root and a complex root; all non-zero.
It should produce the AR parameters & variance di-
rectly as well as the simulated values as output.
Show two simulations (T=200) (on a single page) one
for each of the above cases. List the two sets of pa-
rameters used. In each case ensure that γo ≥ 3.
(b) ♦ Using your mfile simulate an AR(3) with roots
(.9,.1,.6) for T=200. List the true parameter values.
Using least squares regression4 produce estimates for
the 3 parameters, the noise variance as well as stan-
dard errors for the parameters.
Are the estimates within 2 standard errors of the true
values?
(c) ♦ Using your mfile simulate new data (T=100) from
the same model (ii) compute BIC5 and find its mini-
mizing order p∗. Show a single plot of BIC together
with its two components.
Give the parameter estimates corresponding to p∗ and
their standard errors.
Also do a statistical model diagnosis using just the acs
of the residuals. What conclusions do you draw about
the quality of the estimated parameters and model or-
der?
4write your own mfile; don’t use any matlab command for any regres-
sion related computations
5using your own mfile; not matlab’s BIC command
