2026 Prof.Jiang@ECE NYU 214
Lecture V
• The higher dimensional case of general
real symmetric matrices
• Extension and Applications
2026 Prof.Jiang@ECE NYU 215
The General Case
 
We already proved the result with 2.
, let us assume that for each integer
1 , we can find an matrix
which reduces real symmetria matrix
to the diagonal fo
ortho
c
g
rm:

na

o l k
k ij k k
N
By induction
k N O
A a 

 

1

0
0
T
k k k
k
O A O
      

  

2026 Prof.Jiang@ECE NYU 216
The General Case: Goal
 
1
1 ( 1) ( 1)
1
1 1 1
1
We want to find an orthogonal matrix
of order 1, which reduces a real symmetric
matrix to the diagonal form:

0
0
N
N ij N N
T
N N N
N
O
N
A a
O A O

   
  



      

  

2026 Prof.Jiang@ECE NYU 217
Systematic Procedure
 1 ( 1) ( 1)
1
2
1
1
1
1
Let us name the rows of as:

take an eigenvalue and its associated
eigenvectornormalized .
N ij N N
N
N
A a
a
a
A
a
and
x
   



       

 
2026 Prof.Jiang@ECE NYU 218
Systematic Procedure (cont’d)
 
1
1 1
1 2 1
1
By means of the Gram-Schmidt orthogonalization process,
we can form an orthogonal matrix whose first column
is the :
, , ,
Then, as shown in C
give
ase 2, it holds:
n
N
O
x y
O y y y
N


 

1, 11 12
1 1 1

0
,
0
N
T N N
N N N
b
O A O A A
b 


       


2026 Prof.Jiang@ECE NYU 219
Exercise
Can you prove the above identity?
2026 Prof.Jiang@ECE NYU 220
Answer
1 1 1 1
1 1
1 1 1 1
1 2 1 1
1 11
1 2 1 1
1 1, 1
Carrying out the multiplication, we have
, ,

, ,
, ,

, ,
N
N
N N N
N
N N N
N
a x a x
A O
a x a x
x a x a x
x a x a x


  

  

       
       

  


   

2026 Prof.Jiang@ECE NYU 221
Answer (cont’d)
1, 1
1
1 12
1 1 1
Since is an orthogonal matrix, it follows that

0
,
0
N
T N N
N N N
O
b
O A O A A
b 


       


2026 Prof.Jiang@ECE NYU 222
Comment 1
1 1 1
1
1
1 1 1
Furthermore, since must be symmetric,
we have 0, 2,..., 1, and is symmetric.
Thus,
0 0
0
, .
0
T
N
j N
T T
N N N N
O A O
b j N A
O A O A A A


  
       


2026 Prof.Jiang@ECE NYU 223
Comment 2
1
1 1 1
2 3 1
1
Given the identity
0 0
0
,
0
we conclude that the eigenvalues of
be , , , , the remaining eigenvalues
of .
T T
N N N N
N
N
N
O A O A A A
A must
A



       
  



2026 Prof.Jiang@ECE NYU 224
Systematic Procedure (cont’d)
1
1 1
, there is an orthogonal matrix
which reduces to diagonal form. Form the
( 1)-dimensional matrix
1 0 0
0

0
Clearly, is also orthogonal, i.e.,
N
N
N N
T
N N
By induction O
A
N
S O
S S

 

       


1 . NS I 
2026 Prof.Jiang@ECE NYU 225
Systematic Procedure (cont’d)
  11 1 1 1 1
1
1
1 1 1
1
1 1 1
It can be directly checked that

or, equivalently,

with .
0
0
0
0
T T
N N N
N
T
N N N
N
N N
S O A O S
O A O
O O S
  

  

 
      
      


  


  

2026 Prof.Jiang@ECE NYU 226
Formal Statement of the Main
Result
 
1
1
Let be a real symmetric matrix. Then,
it may be transformed into a diagonal form
by using an orthogonal matrix so that

where are the eigenvalues of .
0
0
n n
T
n
n
i i
A
O
O AO
A



      



  

2026 Prof.Jiang@ECE NYU 227
Test for Positive Definiteness
A necessary and sufficient condition for a real
symmetric matrix to be
is that all eigenvalues of are po

sitive.
positive de
A
finiteA
2026 Prof.Jiang@ECE NYU 228
Indeed,
 
 
1
2
1
Recall that a real matrix is if
0, , 0.
,
, where
, where
So, the equivalence property follows readi
posit
ly
ive definite
T n
T T T
i
T T
i n
n
i i
i
A
x Ax x x
Then
x Ax x O O x diag
y y y O x y
y


   
    
   
 

.
2026 Prof.Jiang@ECE NYU 229
Repeated Eigenvalues
1
As shown previously, if a matrix has
eigenvalues, then its associated eigenvectors
are linearly independent.
What if has a repeated eigenvalue of
(algebraic) multiplicit ?
:
y
Ques
A
t
distinc
A
i
t
ons
k
 
 Are there always linearly independent eigenvectors? k
2026 Prof.Jiang@ECE NYU 230
Comment
As shown in , the answer is generally
negative for a real matrix which is symmetric.
However, for a real symmetric matrix, we can
always find linearly independent eigenvectors
for repe
Lecture

IV
any
not
k
ated eigenvalue of multiplicity .k
2026 Prof.Jiang@ECE NYU 231
Indeed,
1
1 1
1
an orthogonal matrix such that

where , , 1,..., .
, the first columns of are of course
linearly independent, and are eigenvectors
associated with
0
0 n
k i
O
AO O
i k n
Now k O

      
         


  


.
2026 Prof.Jiang@ECE NYU 232
In addition,
1
1
Any other eigenvector associated with
of these vectors.
In fact, , with the th column of .
0, 1, , because these eigenvectors
are
linear combina
orthogonal w
o
ith
ti n
n
i
i
i
i
i i
y
is k
y c x x i O
c i k n
x



   


, , 1 .
( Lecture IV)
jx y j k
see
 
2026 Prof.Jiang@ECE NYU 233
Special Case of Cayley-Hamilton Theorem
 
   
 
1
1
As a direct application of the diagonal canonical
form, we have

: 0,
where det ,
:
A
n
n n i
A i
i
n
n n i
A i
i
Any real symmetric matrix satisfies its own
characteristic equation A
I A
A A A




 
         
   

 0, with .A I
2026 Prof.Jiang@ECE NYU 234
Application:
Solving Differential Equations
1 1
1
Solving for the solutions of

an problem for

where is a canonical form of under
a nonsingular transformation
e
:
s
asier
o that a
c
c
c
x Ax
boils down to
y A y
A A
P x y P x
A P AP
 



 



nd .x Py
2026 Prof.Jiang@ECE NYU 235
Exercise 1
     
 
Solve the following initial-value problem:
1 3 1
, 0 .
3 1 0
. ,
?, 0.
x t x t x
i e
x t t
          
  

2026 Prof.Jiang@ECE NYU 236
Exercise 2: Extension to Difference
Equations
0 1
1 2
Find an explicit expression for , 0,1,...,
given that
1, 2 and
, 2,3,...
n
n n n
x n
x x
x ax x n 

  
  
2026 Prof.Jiang@ECE NYU 237
Hint
1 2
1
1
1
Rewrite the second-order difference equation
as:
0 1
, 2,3,...
1
Equivalently,
, 1, 2,...
with
: .
n n
n n
n n
n
n
n
x x
n
x a x
A n
x
x
 

 



             
 
    
2026 Prof.Jiang@ECE NYU 238
Another Extension
When does there exist an orthogonal matrix
which simultaneously reduces two real symmetric
matrices , to diagonal f

o
:
rm?
Questi
O
B
n
A
o
2026 Prof.Jiang@ECE NYU 239
Motivational Problem
Solve the 2nd-order differential equation:
( ) ( ) 0,
where , are matrices.
Note that such equations often occur in
mass-spring probl
symmetric
ems.
n
n n
Ax t Bx t x
A B 
  

 

2026 Prof.Jiang@ECE NYU 240
Basic Result
 
 
A necessary and sufficient condition for the existence
of an orthogonal matrix such that

is that and commute, i.e. .
T
i
T
i
O
O AO diag
O BO diag
A B AB BA
    

Note: See Section 1.3 of the 2013 textbook of Horn and Johnson for extensions
to more than 2 matrices.
2026 Prof.Jiang@ECE NYU 241
Proof of the Necessity
   
Clearly,
and
commute, because is orthogonal.
T T
i iA O diag O B O diag O
O
  
2026 Prof.Jiang@ECE NYU 242
Sufficiency: Sketch of Proof
     
Either or has distinct eigenvalues.
Assume that has distinct eigenvalues. Then,
.
So, , if nonzero, is an eigenvector too,
for the same eigenvalue . In other words,

1: A B
A
Ax x A Bx B Ax Bx
Bx
Bx
Case
 

   
 
is a multiple of . As a result,
, for each pair , .i i ii i
x
Bx x x 
This equality of course
also holds if Bx=0.
2026 Prof.Jiang@ECE NYU 243
Sufficiency: Sketch of Proof
 1 2
, we observe that , have the same
eigenvectors , 1 .
Thus, we can define the orthogonal transformation
matrix as f
1 (con
ollows:
, ,
)
,
t'd
i
n
Now A B
x i n
O
O x
se
x x
Ca
 
 
2026 Prof.Jiang@ECE NYU 244
Sufficiency: Sketch of Proof
1
1
1
repeats times associated with (linearly
independent/orthonormal) eigenvectors , , .
, using previous computation, we have
, 1, 2, , .
In addition, , ,
2 :
i
k
k
i j
ij
j
j j
ij
k
x x
Th
C
en
Bx c x i k
x Bx
ase
c Bx x


 
 



1
.
, consider the linear combination .
i
ji
k
i
i
i
c
Now a x



2026 Prof.Jiang@ECE NYU 245
Sufficiency: Sketch of Proof
 
1 1 1 1 1
1 1
1
1
1 1
We have
( ) .
Thus, if we choose so that
, 1, 2, ,
2 (cont'd) :
, 0
then we have

k k k k k
i j j
i i ij ij i
i i j j i
i
k
ij i j
i
k k
i i
i i
i i
B a x a c x c a x
a
c a ra j k r I C a
B a x r a
Case
x
    

 
         
   
    

    



   
2026 Prof.Jiang@ECE NYU 246
Sufficiency: Sketch of Proof
 
1
1 1
1
1
1

implies is an eigenvalue of , associated
with eigenvector .
On the other hand, is an eigenvalue of
associated with eigenvector
2 (cont'd) :
k k
i i
i i
i i
k
i
i
i
ij
Ca B a x r a x
r
se
B
a x
r C c
a c
 

         


 

 .iol a
2026 Prof.Jiang@ECE NYU 247
Sufficiency: Sketch of Proof
1 1

If is a -dim. orthogonal transformation reducing
into a diagonal form, then
is an orthonormal set
with each being eigenvector for both

2 (end) :
comm n
and .
o
k
k k
k
i
T k
C
z z T x x
z
A B lef
Case
t
       
  as an exercise
Exercise 3
2026 Prof.Jiang@ECE NYU 248
Show how to transform the following matrix into
a canonical diagonal form, by means of an
orthogonal matrix:
1 2 0
2 1 0
0 0 1
M
      
Normal Matrices:
A generalization of real symmetric matrices
2026 Prof.Jiang@ECE NYU 249
* *
Definition: A square matrix is norma said to be ,l
if .
A
AA A A
If is and is a scalar, then also is normal.
If is and , then also is normal.
Every unitary matrix is normal.
Every real symmetric or skew-symme
normal
tric m
normal
atrix is normal.

A A
A B A B
 





Every Hermitian or skew-Hermitian matrix is normal.
Exercise 4
2026 Prof.Jiang@ECE NYU 250
Let , be constants. Show that is normal
and has eigenvalues .
a b
a b
b a
a ib
   

Exercise 5
2026 Prof.Jiang@ECE NYU 251
11 12
22
11 22
A matrix is if = .
Show that a matrix
is conjugate normal
0
if and on
block-upper-tr
ly if its diagonal blocks , are conjugate
norma
conjug
iangular
l,
ate normaln nA AA A A
A A
A
A
A A
  
    

12 and 0. In particular, an upper triangular
matrix is conjugate normal iff it is diagonal.
A 
See the text (2nd ed.) by Horn-Johnson, p. 268.
2026 Prof.Jiang@ECE NYU 252
Homework #5
1 2 3
1 2 3
1. Using the Gram-Schmidt process find a set
of mutually orthonormal vectors , , ,
based on:
1 0 0
0 1 0
, , .
0 0 1
1 1 1
u u u
x x x
                                  

学霸联盟