2026 Prof.Jiang@ECE NYU 117
Lecture III
Key issues:
• Eigenvalues, eigenvectors and the
characteristic polynomial of a square matrix
• Similarity
2026 Prof.Jiang@ECE NYU 118
Eigenvalue and Eigenvector
 
t
Given a square matrix , the set of s
are such that the
, or equivalently 0
characteri

stic equati
r
on
ei

genv
l
eig
t
enva
o
ec
has a nonze o so u i n 0.
Such a vector is called an
lue
n
n n A
Ac c I A c
c
c


    



 associated
with eigenvalue .
or

Definition:
As a result, for any eigenvalue,
   det 0, .I A A     
2026 Prof.Jiang@ECE NYU 119
How to compute an eigenvalue?
   
1
1 1
characterist
The eigenvalues
ic polynomia
of a matrix are
the roots of its :
det

l
.
n n
n n
n n
A
I A



 
    
       


So, in total there are n eigenvalues (possibly repeated
and/or complex-valued).
They are the roots of the characteristic polynomial of A:
2026 Prof.Jiang@ECE NYU 120
Comment on Characteristic Polynomial
 
  11 1
, det 0 implies, by means of permutation:
det 0.
Indeed
n n
n n
I A
I A  
  
          
2026 Prof.Jiang@ECE NYU 121
Comment 1
For any given eigenvalue λ, the eigenvectors
are
1) nonzero solutions to the homogeneous
equation:
(λI - A)c=0.
2) non-unique.
Clearly, any vector of the form µ x c for a
nonzero scalar µ is still an eigenvector.
2026 Prof.Jiang@ECE NYU 122
Comment 2
For any given eigenvalue λ, the associated
eigenvectors c are nonzero elements of the
null-space of (λI - A), i.e.:
So, the maximum number of linearly
independent eigenvectors is equal to:
n – rank(λI - A).
  , 0.c Null I A c  
2026 Prof.Jiang@ECE NYU 123
An Example
1 2
1 2
1 0
For the identity matrix , the eigenvalues
0 1
are 1, for which two
eigenvectors are:
1 0
, .
0 1
Othe
linearly indep
r choices of i
end
ndependent eigenvectors are:

e

t

n
A
c c
    
   
          
1 21 3, .
2 1
c c          
2026 Prof.Jiang@ECE NYU 124
A Useful Observation
A square matrix of dimension is singular
it has a zif and on ero eigenly i vaf lue.
A n
2026 Prof.Jiang@ECE NYU 125
Proof

Assume that is singular. Then, there is
a nonzero solution 0 to the equation
(*) 0, or equivalently, 0 .
So, 0 is an eigenvalue.
If 0 is an eigenvalue, then t
:
he
:
A
x
Ax Ax x
Necessity
Sufficiency



 



re is an eigenvector
0, which is a solution to (*).
Thus, must be a singular matrix.
x
A

2026 Prof.Jiang@ECE NYU 126
Exercise
Find the eigenvalues and the associated eigenvectors
of the following matrix
1 1
.
1 1
M     
2026 Prof.Jiang@ECE NYU 127
More about Eigenvalues
A real square matrix can complex have eigenvalues.
1 2
For example, the eigenvalues of
0 1

1 0
are , .j j
   
    
2026 Prof.Jiang@ECE NYU 128
More about Eigenvalues
A real square matrix can multiplehave eigenva . lues
   
1 2
1
2
For example, the eigenvalues of
1

0 1
are 1 for any .
We say that 1 is an eigenvalue of multiplicity 2.
In this case, the characteristic polynomial is
det 1 .
a
A
a
I A
    
    
 
    

2026 Prof.Jiang@ECE NYU 129
More about Eigenvalues
       1 21 2
1
More generally, the characteristic polynomial of
a matrix takes the form
det
, , are
an eigenvalue of (algebr
different, wi
aic) m
th ,
ultiplicit
r
n n
m m m
r
r i
i
i
A
I A
where m n



        


 



 
1
1
.
When and 1, the matrix is
y
dis
said to
have eiget nvalinc ue .t s
n
n
i
i i
r A
m
n m m

   


2026 Prof.Jiang@ECE NYU 130
More about Eigenvectors
1
1 2
1
Back to the example of . It has an eigenvalue
0 1
1 of (algebraic!) multiplicity 2, for any .

For 0, the associated eigenvectors ar distine :
1 0

Cas

e 1:
,
0
ct

a
A
a
a
c c
     
  

    

.
1

   
2026 Prof.Jiang@ECE NYU 131
More about Eigenvectors
1
1
e
Ca e
1
Back to . It has an eigenvalue 1
0 1
of (algebraic) multiplicity 2, for any .

For 0, there is distinct
1
eigenvector:
0
. .,
only one
all other eigenv
s 2:
a
A
a
a
c
i e
      


    

1ctors take the form , for some
scalar 0.
r c
r


Its “geometric” multiplicity is 1
2026 Prof.Jiang@ECE NYU 132
Exercise
Compute the eigenvalues of the following matrix
1 1 1
1 1 1 .
1 1 1
Can you find the eigenvectors associated with
each eigenvalue?
M
      
2026 Prof.Jiang@ECE NYU 133
Exercise
For any matrix ,
1) can any eigenvalue of be complex?
2) can the largest eigenvalue of be negative?
n n
T
T
A
A A
A A

2026 Prof.Jiang@ECE NYU 134
A General Result
1
1
Assume has eigenvalues , ..., .
,
(1) must have eigenvectors
, , .
(2) In addition, each eigenv
li
ec
nearly independen
d
tor associated with
is
istin
apart f
ct
t
unique
n n
n
n
j
j
A n
Then
A n
c c
c
  



rom a nonzero scalar multiplier.
2026 Prof.Jiang@ECE NYU 135
Proof of Statement 1
 
1
1
Let 0, , 0 be eigenvectors satisfying
, for 1, 2, , .
We prove the statement by contradiction.
Assume that are linearly dependent. Let
be the le positive integer suas cht
n
i i
i
ni
i
c c
Ac c i n
c k n
 
  



 
1
1 2
1 2
that of the 's are
dependent. Without loss of generality, assume that
are dependent, th not aat ll is, zer ,

o
0.
ki
ii
k
k
k c
c
c c c
 
      
2026 Prof.Jiang@ECE NYU 136
Proof of Statement 1 (cont’d)
 
   
1 2
1 2
1 1
1 1 1 1
, 0.
Thus, 2 and 0 (otherwise, contradiction
with being the least).
, multiply the eq. b
not al
y leads to:
0
which,
l zero
all
in turn, i
k
i k
i
k
k
k k k k
c c c
k
k
Now A I
c c  
     
  

       


 
1
1
mplies that are dependent.
A contradiction.
i
i
k
c 

2026 Prof.Jiang@ECE NYU 137
Proof of Statement 2
 
1
1
1
We must show the " " of :
, 0 , with 0.
As it was proved in statement (1), are
linearly independent and thus form a basis. So
uniqu
,
.
Multiplyi
eness
ng t
i
i
i
ni
i
n
n
i
i
c
Ac c c c c
c
c c cc

       
       
 
   11 1
he above eq. by gives:
0
Therefore: 0, .
In other words, as wished
0
, .
i
n
i n i
k
i
n
i
A I
c c
k
c
i
c
 
            
 

 

 
2026 Prof.Jiang@ECE NYU 138
Corollary
 
 
1 2
1
Under the above conditions, define matrix

which is nonsingular & implies
.
In this case, we say t similar
similar
hat is ,
while is it y maa
o
t
t

n
i
i
P c c c
P AP diag
A diag
P



   


 
.
Denote . is called diagonali
rix
zable" "iA diag A
2026 Prof.Jiang@ECE NYU 139
Remark 1
 As we will see in Lecture IV, is
a canonical form for
1) the class of matrices having distinct eigenvalues;
and
2) the class of real symmetric matrices,
which may have repeated eigenvalues.
idiag 
2026 Prof.Jiang@ECE NYU 140
Remark 2
   
   
 
1
1
1 1
1
Indeed, , such that .
It then follows that
de
Any two similar matrices and must
have the
t det
det det det det
det ,
because det det d
same eigenvalues.
A B
A B P B P AP
B I P AP I
P A I P P A I P
A I
P P
 
 



 

  
  
   
 


1et 1.P P 
2026 Prof.Jiang@ECE NYU 141
Questions
Are you ready for some tricky questions?
2026 Prof.Jiang@ECE NYU 142
Question 1
   
 
3 3
3
If the set of all eigenvalues of , or
the 1, 2,spectrum 3 ,
what is ?
A
A
A A



 


2026 Prof.Jiang@ECE NYU 143
A General Result
 
 
 
0
0
0
For any polynomial , and any
matrix , denote ( ) , with .
If , is a pair of eigenvalue and eigenvector
of , then ( ), is a pair of eigenvalue
and eigenvector of ( )
k
i
i
i
k
i
i
i
p t a t n n
A p A a A A I
A p
x
x
p A




 


 
.
(Its proof is left as an exercise.)
“Matrix Polynomial”
2026 Prof.Jiang@ECE NYU 144
Question 2
2For any matrix , that is, ,
what are the possi
idempo
ble ei
ten
gen
t
values?
A A A
Idempotent Matrix: 2x2 case
2026 Prof.Jiang@ECE NYU 145
2
2
If is idempotent, then
,
, implying 0, or 1
, implying 0, or 1
.
a c
A
b d
a a bc
b ab bd b d a
c ca cd c d a
d bc d
    
              
Examples: Idempotent Matrix
2026 Prof.Jiang@ECE NYU 146
0
(1) is idempotent, if , 0,1.
0
1 cos sin1
(2) .
sin 1 cos2
a
A a d
d
A
 
 
    
    
2026 Prof.Jiang@ECE NYU 147
Answer
An idempotent matrix can only have 0 or 1
as its eigenvalues.
2026 Prof.Jiang@ECE NYU 148
Question 3
A matrix is such that 0 for a positive
integer . Such a smallest is called the
.
What are the eig
nilpotent
ind
envalues of
ex of nilp
a nilpo
o
t
t
ent matrix ?
ency
qA A
q q
A

2026 Prof.Jiang@ECE NYU 149
Answer
All eigenvalues of a nilpotent matrix are 0.
Indeed, if , 0, then,
using 0, we obtain 0
which, in turn, implies 0.
q q
Ax x x
A x



 
 

Examples: Nilpotent Matrix
2026 Prof.Jiang@ECE NYU 150
0 * *

(1) *
0 *
0 0 0
5 3 2
(2) 15 9 6
10 6 4
M
M
       
      

 

Equivalence Relation: Nilpotent Matrix
2026 Prof.Jiang@ECE NYU 151
The following statements are equivalent:
(1) is nilpotent.
(2) The minimal polynomial of is , for some .
(3) The characteristic polynomial of is .
(4) The only eigenvalue of is 0.
n n
q
n
M
M s q n
M s
M



2026 Prof.Jiang@ECE NYU 152
Exercise
Compute the algebraic and geometric multiplicities
of the eigenvalue 2 for the matrix
2 1 0 0
0 2 0 0

0 0 2 1
0 0 0 2
A
 
       
2026 Prof.Jiang@ECE NYU 153
Useful Identities about Matrix Eigenvalues
 
1
1
For any matrix , we have
( ) ,
det .
n
i
i
n
i
i
n n A
trace A
A









See p.42 of (Horn-Johnson, 1st ed., 1985),
or p. 50 of (Horn-Johnson, 2nd ed., 2013).
Similarity
2026 Prof.Jiang@ECE NYU 154
Two matrices A and B are said to be similar, if
1 , for some invertible matrix
Notatio
.
n:
B P AP P
A B


 1
The set of all similar matrices to a given square matrix :
: is invertible
Similar matrices are just different basis representations of
a single li
No
near map
te
ng.
:
pi
A
S P AP P
Similarity: Physical meaning
2026 Prof.Jiang@ECE NYU 155
   
   
1
1 1 2 1
1 1 1
1 1
2
Let : be a linear transformation, and
, , , , , be two bases for .
Denote
, ..., , with ... .
, by linearity,
...
For any basis of ,
n n
n n nB
n n
j
T V V
B v v B w w V
x col x v v
Then
Tx Tv Tv
B V
Tv
   
 

 
   
  
 
 
     
2
2 2
1
1
1
, ..., ,
, ..., .
j njB
n
j j ij nB B nxn
j
col t t
Tx Tv t col  

  
   

Similarity: Physical meaning
2026 Prof.Jiang@ECE NYU 156
     
 
2 2
1
1
1 2
1 2
, ..., .
It is important to note that the matrix depends on
and the choice of the bases and , but .
Define the - as:

not
n
j j ij nB B nxn
j
ij nxn
Tx Tv t col
t T
B B x
B B basis representation of T
  

   
       
     
 
2 1 2 2
22 1 1
1 1
1
1 2
1
, ...,
, = , .
For the special case when ,
is called the representatio .n of
B ij nB B Bnxn
BB B B
B B
T t Tv Tv
B T
So Tx T x x V
B B
T
   
 


Similarity: Identities
2026 Prof.Jiang@ECE NYU 157
       
       
     
2 11 1 2 2 2 1
2 2 1 12 1 1 2
1 1 22 1 2
1
For the identity linear transformation , ,
it can be shown that
, ,
= .
In other words,
, where , , .
B n B nB B B B B B
B B B BB B B B
B B BB B B
Ix x x V
I I I I I I
and
T I T I
B P AP P I A T B T
  
 
   
For a proof, see (Horn & Johnson, 2nd ed, 2013, page 40).
2 1- change of basis matrixB B
Exercise
2026 Prof.Jiang@ECE NYU 158
1
an invertible matrix such that
1 1 1
0 2 2
0 0 3
is diagonal.
Find P
P P
     
Exercise
2026 Prof.Jiang@ECE NYU 159
1
Let , be similar:
, for some .
For any eigenvector of , show that
is an eigenvector of .
n n n n
n n
A B
A P BP P
x A
Px B
 
 
 
 
 

2026 Prof.Jiang@ECE NYU 160
Homework #3
1. Give all the solutions of the system
1 2 3 10 13
4 5 6 .11 14
7 8 9 12 15
2. Prove that the following eq. has no solution:
1 3 1
.
2 6 3
x
x
              
         
2026 Prof.Jiang@ECE NYU 161
Homework #3
1 2
0 1 2
1 2
3. Find a least-squares fit

for the data:
1 1 0
0 2 0
, , .
0 3 0
0 4 1
b x x a x a
b a a
  
                                
2026 Prof.Jiang@ECE NYU 162
Homework #3
4. Find independent eigenvectors for
1 2
.
3 1
1
Can you express as a linear combination
2
of these eigenvectors of ?
A
x
A
    
    

学霸联盟