MAT 653 Homework 4
Wei Li
Due Nov 14th, 11:59pm ET
General Instructions
• You must upload your solutions to the right subfolder in your shared folder (say HW1, HW2, etc.) in
the Google Drive folder before the deadline.
• Your solution must include (1) a PDF-file solution, (2) a standalone R code that can replicate your
computation results, (3) executable .Rmd file with all necessary pictures that compiles your PDF
file. I will grade your answer solely based on your PDF file. Therefore, please include only necessary
information in your PDF solution for grading. The rest files (R code and .Rmd) are supporting files
that I will loosely examine when needed.
• In your R simulation, be sure to include set.seeed to fix the random sequence. So people can replicate
your results.
Problem 1
Consider the mixture of two normal distributions discussed in the class.
Xi ≥ 14N(µ1, 1) +
3
4N(µ2, 1)
Set a seed number set.seed(2021). Generate 400 i.i.d. observations from above density (setting the truth
µú1 = 0, µú2 = 2.5). Use EM algorithm discussed in the class to obtain the estimate (µ1, µ2). Try two set of
starting values
(i) (µ1 = 0, µ2 = 0)
(ii) (µ1 = 3, µ2 = 1).
Plot the heatmap and contours for the log likelihood function, and make plots to demonstrate how the
estimates iterate for both cases. What are your final estimates for µ1 and µ2? Does your choice of starting
values matter very much for the final results?
Problem 2
In Christian Robert and George Casella’s book, the simulated annealing algorithm is applied to find the
maximizer of ≠h(x, y) where
h(x, y) =(x sin(20y) + y sin(20x))2 cosh(sin(10x)x)
+ (x cos(10y)≠ y sin(10x))2 cosh(cos(20y)y)
See (Example 5.10). The simulated annealing algorithm modified the original algorithm in example 5.9 with
modification
prop=the [iter, ]+scale[iter]*rnorm(2)
1

scale=min(0.1, 1,5 *factor*sqrt(temp[iter]))
Note the two new tweaks from our R example in the class. One is the use of scale which modifies the
variance of the proposal distribution. Another tweak is the use of factor which is to modify the magnitude
of scale according to how often the proposals are accepted which is indicated by ace. Note also the temp
will also modify the scale as iteration goes on.
In this problem, you are asked to use the modified (improved) version of simulated annealing algorithm to
replicate the Figure 5.10 in the textbook, using
(1) Tt = (0.95)t
(2) Tt = 1/(10(t+ 1))
(3) Tt = 1/ log(1 + t)
(4) Tt = 1/(10

log(1 + t))
Set the seed number to 2021 for each of case. You can modify the R code in the class (as the code example
in the textbook has typos). Note that you are not required to produce the exact same plots, but you may
observe similar pattern.
Problem 3
In this problem, you are asked to use stochastic gradient algorithm to solve for Problem 1 (maximization
problem). Use the exact same generated dataset obtained in Problem 1. To do so, you may modify the R
code given in the class (note the code example in the textbook has typos). For the same two set of starting
values
(i) (µ1 = 0, µ2 = 0)
(ii) (µ1 = 3, µ2 = 1).
Plot the heatmap and contours for the log likelihood function, and make plots to demonstrate how the
estimates iterate for both cases. What are your final estimates for µ1 and µ2? Does your choice of starting
values matter very much for the final results?