High Dimensional Data Analysis Assignment 3: S2, 2023
Department of Econometrics and Business Statistics, Monash University
Due Date: 13th October 2023 at 4:30PM
Questions for ETF3500 and ETF5500 students
1. Consider the case where Y is an n× p data matrix containing n observations on p-variables. The data
matrix has been demeaned. Use this information to answer the following questions. (4 marks).
• S denotes the sample covariance matrix of Y. Using matrix notation, state the sample covariance
matrix of Y. Be sure to clearly define any quantities needed to state S.
• What is the dimension of the sample covariance matrix of Y.
• What is the dimension of an arbitrary eigenvector associated with S?
2. Let w be the eigenvector of S corresponding to the second largest eigenvalue. In addition, let u be
any arbitrary column vector, whose dimension is the same as that of w. Discuss whether the following
matrix products are conformable (5 marks).
• Yw
• Yw′
• ww′
• u′w
• S′Y
3. What will be the dimensions of b where b = (Yw)′? (1 marks)
4. Assume that w′w = 1. Describe what is contained in b. What do we call this? (1 marks)
5. From this question onwards, let p = 2. Let w1 and w2 denote the eigenvectors of S that satisfy
w′1w1 = 1 and w′2w2 = 1. Prove that w1w′1 +w2w′2 =
[
1 0
0 1
]
.
Hint: By property of the eigenvectors, if w1 =
[
a
b
]
, then w2 will either take the form w2 =
[
b
−a
]
or w2 =
[ −b
a
]
.
(4 marks)
6. Let w1 and w2 be eigenvectors of S with eigenvalues λ1 and λ2, respectively, where λ1 > λ2. Consider
now the first principal component vector c1 = (c11, . . . , cn1)′ and second principal component vector
c2 = (c12, . . . , cn2)′. Use the result in question 5. above to prove that
Y = c1w′1 + c2w′2
(5 marks)
1
Questions for ETF5500 students only
7. The main purpose of this question is to link PCA to the construction of the SVD for a data matrix Y.
The following formula presents the generic singular value decomposition of the data matrix.
Y = d1u1w′1 + d2u2w′2.
We know that the singular values of the matrixY are equivalent to the square root of the eigenvalues
of the matrix Y′Y. Use this information, and the expression provided in questions 5 and 6 above, to
write down the terms d1, d2, u1 and u2 solely in terms of c1, c2, w1, w2, λ1, λ2 and n.
(5 marks)
8. Let us construct a new variable A, whose (n× 1) vector of observations is constructed by a = 1√
λ1
c1
and a new variable B, whose (n× 1) vector of observations is constructed by b = 1√
λ2
c2. The terms
c1, c2, λ1, λ2 are formally defined in Question 6 above. Let us collect these two new variables into a
new (n× 2) data matrix X = [ a b ]. Prove that the sample covariance matrix of X is an identiy
matrix.
(5 marks)
Submission
The assignment is an individual assignment. A soft copy should be submitted with a cover page on the front.
All assignments should be submitted via Moodle.