2026 Prof.Jiang@ECE NYU 253
Lecture VI
Extensions to Complex Matrices, in particular
Hermitian Matrices.
Key Notions:
* Unitary matrices
* Unitary equivalence
* Schur’s unitary triangularization
* QR factorization
* Congruence and simultaneous diagonalization
2026 Prof.Jiang@ECE NYU 254
Orthogonality Between Complex
Vectors
*
1 1 2 2
Given any pair of ( ) vectors , ,
the inner product is defined as
,
.
They are said to be , if
,
o
0
rthogon l
.
a
n
n n
complex x y
x y y x
x y x y x y
x y

   




2026 Prof.Jiang@ECE NYU 255
Facts about the Inner Product
It can be easily checked that the inner product enjoys
the following properties:
, , , , , , .
, , , , scalar.
0, ;
,
0, if and only if 0.
n
n
x y z x y x z x y z
x y x y
x
x x
x
  
     
   
     



2026 Prof.Jiang@ECE NYU 256
Orthogonal & Orthonomal Sets of
Vectors
A set of vectors is said to be , if
, 0, 1 , , .
A set of vectors is said t
orthogonal
orthono be
if, add
orma
itionally, : , 1, 1 .
l

i n
n
i j
i
i i i
x
x x i j k i j
x
x x x i k
 
    
 
    


2026 Prof.Jiang@ECE NYU 257
Remark
 
1
Any orthogonal set of vectors
can be made an orthonormal set, by defining
1

no
: , 1 .
nzer
,
o
ki
i
i i
i i
y
x y i k
y y

   
2026 Prof.Jiang@ECE NYU 258
Fundamental Results
1) Any orthogonal set of nonzero vectors
is linearly independent.
2) Any orthonormal set of vectors is
linearly independent.
2026 Prof.Jiang@ECE NYU 259
Unitary Matrix
* *
A matrix is said to be
if . (Recall that )
Of course, a real orthogonal matrix
is unitary, but the converse is not true.
C
unita
an you find some examp
r
e
y
l s?
n n
n n
TU U
U
U U I
O








Complex Orthogonal Matrix
2026 Prof.Jiang@ECE NYU 260
complex orthogA matrix is said to be , if:

ona

l
.
n n
T
A
A A I



Remark:
A complex orthogonal matrix is unitary if and only if
it is real.
2026 Prof.Jiang@ECE NYU 261
Equivalent Characterizations
* 1
*
*
The following are equivalent:
is unitary;
is nonsingular and ;
;
is unitary;
The columns of form an orthonormal set;
The rows of form an orthonormal set;
For any
U
U U U
UU I
U
U
U
x


 
 



  * *, satisfies .n y Ux y y x x 
Exercise
2026 Prof.Jiang@ECE NYU 262
 
1) For any given real parameters , 1 ,
is always unitary.
2) Any diagonal unitary matrix can always be put into
the above form.
3) Any diagonalizable unitary matrix can be transformed
to
k
i
j
i n
U diag e 
  

the above form.
Are the following statements true or false?
2026 Prof.Jiang@ECE NYU 263
Question
How to apply a unitary matrix, instead of
a real orthogonal matrix, to transform a
Hermitian matrix into a canonical
diagonal form?
2026 Prof.Jiang@ECE NYU 264
Review: Canonical Form of a Real
Symmetrical Matrix
 
1
1
Let be a real symmetric matrix. Then,
it can be transformed into the diagonal form
by using an orthogonal matrix so that

where are the eigenvalues of .
0
0
n n
T
n
n
i i
A
O
O AO
A



      



  

2026 Prof.Jiang@ECE NYU 265
Extension
*
*
It is possible to generalize this important result to
(possibly complex) matrices , Hermitian
unitary matr
. .,
.
In this case, we use , instead of
orthogonal matrices,
ices
. .,
.
H i e
H H
U
i e
U U I


2026 Prof.Jiang@ECE NYU 266
Examples
1 2
The matrix is Hermitian.
2 3
1 2
The matrix is Hermitian,
2 3
but is a complex symmetrical matri
not
x.
i
i
i
i
     
     
2026 Prof.Jiang@ECE NYU 267
Eigenvalues of Hermitian Matrices
The eigenvalues of a Hermitian matrix are real,
and eigenvectors associated with distinct
eigenvalues are orthogonal.
2026 Prof.Jiang@ECE NYU 268
Canonical Transformation
1
*
If is a Hermitian matrix, there exists a unitary
matrix such that
.
In particular, becomes a real orthogonal matrix
when is a real symmetric matrix.
0
0 n
H
U
U HU
U
H
      

  

2026 Prof.Jiang@ECE NYU 269
Idea of Proof
As in the case of real symmetric matrices, we
use the Gram-Schmidt Orthogonalization
Process, noting the following:
1
For complex vectors , ,
the inner product is defined
as follows:
, .
n
n
T
i i
i
x y
x y y x x y




 
2026 Prof.Jiang@ECE NYU 270
Exercise
1 2
1
2
1
Compute the eigenvalues , of
1 2

2 3
and find a unitary matrix that
0
reduces to the diagonal form .
0
( : use , = for vectors
, in the orth
Hint
ogonaliz
n
i i
i
i
H
i
U
H
x y x y complex
x y

 
     
   

ation process.)
2026 Prof.Jiang@ECE NYU 271
Schur’s Unitary Triangularization
1
*
For square, necessarily Hermitian, matrix
, there is a unitary matrix for which
*
0

*
0
* being zero or nonzero scala
not
rs.
*
0 n
any n n
A U
U AU
with

       



2026 Prof.Jiang@ECE NYU 272
Algorithm
   
1
1
2
1 2
Take a normalized eigenvector of
associated with an eigenvalue , and
find 1 vectors , , so that
, , ,
1:
are linearly independent.
n
n
x A
n y y
x y
Step
y

 

2026 Prof.Jiang@ECE NYU 273
Algorithm
1 2
1 2
1 2
1
Apply the Gram-Schimidt orthonormalization
procedure to , , , to produce an
orthonormal set , , , .
Define , , , which, c

learly,
2 :
n
n
n
Step
x y y
x z z
U x z z   


 is
a unitary matrix.
2026 Prof.Jiang@ECE NYU 274
Algorithm
 
1 2
1
( 1) 11*
1 1 1
1
1 2
Under , , , ,
*
, with .
0
Of course, has eigenvalues , , .
2 (cont'd) : n
n n
n
U x z z
U AU A
A
tep
A
S

 
  
   
    



2026 Prof.Jiang@ECE NYU 275
Algorithm
 
 
( 1) 1
1
2 3
1 1
1
( 1) 12 3
2 1 1
2*
2 1 2
2
For , apply Steps 1-2
to arrive at an orthonormal set , , ,
and a unitary matrix
, , ,
so that
*

3

:
0
n n
n
n
n nn
A
x z z
U x z
Ste
z
U A
A
p
U

  

  


   





 
 ( 2) 2
2, with
n nA   
  

2026 Prof.Jiang@ECE NYU 276
Algorithm
   
2 1 2
2
1
2
*
1 2 1 2
( 2 ) 2 2
It is easy to check that,
1 0
and
0
are both unitary. In addition,
* * *
0 *
4
*


:

n n
n
V U V
U
U V A U V
St p
O
e
A



 
    
                

2026 Prof.Jiang@ECE NYU 277
Algorithm
( 1)( 1)
Continuing these steps to arrive at the
last step, where we have produced
unitary matrices , and
, 2,3,
:
, 1

n i n i
i
n n
i
Last t p
V
e
U
n
S
i
   


  

 
1 2 1
1
*
so that
, and
* *
.
0
n
n
U U V V
U AU


 
       


   

2026 Prof.Jiang@ECE NYU 278
Some Applications of Schur’s
Theorem
• Useful for solving algebraic, differential or
difference linear equations.
Do you know why?
2026 Prof.Jiang@ECE NYU 279
Applications of Schur’s Theorem
• Cayley-Hamilton Theorem
 
   
 
1
1
1
1
Let be the characteristic polynomial of ,
, det .
Then, : 0.
A
n n
A n
n n
A n
p A
that is p I A
p A A A I

     
 


     
    


See the textbook of Horn & Johnson (2nd ed., 2013),
pp. 109~110.
2026 Prof.Jiang@ECE NYU 280
Comment
Cayley-Hamilton Theorem is extremely
important in linear systems theory.
2026 Prof.Jiang@ECE NYU 281
Technical Remark
1
1
1 1
For any square matrix , for any integer ,
there exist constants , , such that
, .
i in
i n
i in in
n n A i n
c c
A c A c A c I i n 
 
     


2026 Prof.Jiang@ECE NYU 282
Exercise
2 3 4
1
3 2
Consider the matrix .
1 0
Use Cayley-Hamilton Theorem to
express , , as linear combinations
of , .
Use Cayley-Hamilton Theorem to find
the inverse .
A
A A A
A I
A
    


2026 Prof.Jiang@ECE NYU 283
QR Factorization
For any (possibly nonsquare) matrix ,
, , such that
The columns of form an orthonormal set,
and is an upper triangular matrix;
.
If, in addition, is nonsingula
n m
n m m m
A
with Q R
Q
R
A
n
QR
m
A

 

  

 

 
r, then the diagonal
entries of are positive. Moreover, in this case,
and are unique.
R
Q R
2026 Prof.Jiang@ECE NYU 284
Remark
The factors Q and R may be taken real, if
A is a real matrix.
Proof: See the textbook, pp.89~90, for the
constructive procedure closely tied to the
Gram-Schmidt (G-S) algorithm.
2026 Prof.Jiang@ECE NYU 285
An Example
What is the factorization of
1 0

2 3
QR
A     
2026 Prof.Jiang@ECE NYU 286
Solution
 
 
1 2
1 1 1
2 2 1* 2 1
1 0
For simplicity, denote : .
2 3
1 2
Then, let / and,
5 5
like in the G-S process, compute
6 3

5 5
T
T
A a a
q a a
y a q a q
    
     
      
2026 287
Solution (cont’d)
 
 
2 2 2
1 2
1
2 1
Now, let / .
5 5
which, by construction,
is orthonormal. Then, , (with =0 )
can be determined according to the general formula:
, 1, 2,...,
T
ij kj
j
j k
kj
k
q y y
Set Q q q
R r r k j
a r q j m

     

  
  m = 2, here
R is upper-triangular.Prof.Jiang@ECE NYU
2026 Prof.Jiang@ECE NYU 288
Solution (end)
11 21 12 22
6 3
So, 5, 0, , .
5 5
6
5
5
That is:
3
0
5
It is directly verified that .
r r r r
R
A QR
   
       

2026 Prof.Jiang@ECE NYU 289
Application to
Cholesky factorization
*
*
By means of factorization, any matrix
taking the form , with ,
can be written as:
, with lower triangular.
Moreover, this factorization is unique,
if is nonsingu
n n
n n
n n
QR B
B A A A
B LL L
A




 
 



lar.
Indeed, it suffices to write to obtain .A QR L R 
B: Positive semi-definite
2026 Prof.Jiang@ECE NYU 290
QR Numerical Algorithm
This is a powerful tool for computing the
eigenvalues of a matrix.
2026 Prof.Jiang@ECE NYU 291
QR Numerical Algorithm
0
0 0 0
1 0 0
1 1 1
1
For any given , factorize

Define , and factorize

Continuing this process, we have
1,
1:
2 :
n n
k k k
k k k
Step
St
A
A Q R
A R Q
A Q R
A Q R
k
e
A Q
p
R






   

2026 Prof.Jiang@ECE NYU 292
Proposition
0
0
Each is unitarily equivalent to , and thus
they have the same eigenvalues.
If has distinct eigenvalues, then converges
to an upper triangular matrix.
k
k
A A
A A


2026 Prof.Jiang@ECE NYU 293
A Numerical Exercise
Use MATLAB simulation to validate
the algorithm for the matrix
1 0
.
2 3
QR
A     
Congruence
2026 Prof.Jiang@ECE NYU 294
*
Consider two matrices , .
(1) is said to be , if
for some nonsingular matrix .
(2) is said to be ,
if for some nons
*
, or
ingular m
T
n n
T
congruent to
congruent congruent t
A B
B A B SAS
S
B A
B SAS
o




atrix .S
Notice that both congruence are equivalence relations.
(Horn-Johnson, 2nd ed., 2013; p. 281)
Inertia
2026 Prof.Jiang@ECE NYU 295
      
 
 
 
0
0
Consider a Hermitian matrix .
Its is defined as the ordered triple:
( ) , ,

the number of positive eigenvalues of ;
the number of negative eigenval
inerti
ues of ;
a
n nA
i A i A i A i A
where
i A A
i A A
i A

 







the number of zero eigenvalues of .A
Sylvester’s Law of Inertia
2026 Prof.Jiang@ECE NYU 296
Hermitian matrices , are *congruent
if and only if they have the same inertia,
i.e., the same number of positive eigenvalues and
the same number of negative eigenvalues.
n nA B 
For the proof, see (Horn-Johnson, 2nd Ed., 2013, p. 282)
Simultaneous Diagonalization
2026 Prof.Jiang@ECE NYU 297
* *
Consider two Hermitian matrices , .
There is a unitary matrix and real diagonal
matrices , such that ,
is Hermitian, that is, .
n n
n n
A B
U
M A U U B UMU
iff AB AB BA




   



See (Horn-Johnson, 2nd Edition, 2013, page 286.)
2026 Prof.Jiang@ECE NYU 298
Homework VI
2
1. Transform the following Hermitian matrix
0 2 1
2 5 6
1 6 8
into a diagonal form.
2. If a (real) Hermitian matrix is positive definite,
prove that ,
for a positive
H
H
H P
       

definite matrix .P

学霸联盟