Real Spectral Theorem

Andrew Thangaraj

Aug-Nov 2020

Recap

  • Vector space \(V\) over a scalar field \(F\)
    • \(F\): real field \(\mathbb{R}\) or complex field \(\mathbb{C}\) in this course
  • \(m\times n\) matrix A represents a linear map \(T:F^n\to F^m\)
    • dim null \(T+\) dim range \(T=\) dim \(V\)
    • Solution to \(Ax=b\) (if it exists): \(u+\) null\((A)\)
  • Four fundamental subspaces of a matrix
    • Column space, row space, null space, left null space
  • Eigenvalue \(\lambda\) and Eigenvector \(v\): \(Tv=\lambda v\)
    • There is a basis w.r.t. which a linear map is upper-triangular
    • If there is a basis of eigenvectors, linear map is diagonal w.r.t. it
  • Inner products, norms, orthogonality and orthonormal basis
    • There is an orthonormal basis w.r.t. which a linear map is upper-triangular
    • Orthogonal projection: distance from a subspace
  • Adjoint of a linear map: \(\langle Tv,w\rangle=\langle v,T^*w\rangle\)
    • null \(T=\) \((\)range \(T^*)^{\perp}\)
  • Self-adjoint: \(T=T^*\), Normal: \(TT^*=T^*T\)
    • Eigenvectors corresponding to different eigenvalues are orthogonal
  • Complex spectral theorem: \(T\) is normal \(\leftrightarrow\) orthonormal basis of eigenvectors

Real vs complex vector spaces

Concept Real Complex
Linear maps - -
Fundamental spaces rowspace \(=\) \((\)null\()^{\perp}\) Need not hold
Eigenvalues Need not be real At least one complex exists
Inner product Dot product Conjugate dot product

Example (\(A\): \(n\times n\) real matrix with real eigenvalue \(\lambda\))

Then, eigenvector \(\in\) null\((A-\lambda I)\) and is real

Example (normal, but not self-adjoint)

\(A=\begin{bmatrix} 0&1\\ -1&0 \end{bmatrix}\)

\(\lambda = i,-i\)

Eigenvectors: \((i,1)\), \((-i,1)\)

Eigenvalues of self-adjoint operators

\(V\): vector space over \(\mathbb{R}\)

\(T:V\to V\), self-adjoint

Eigenvalues of \(T\) are real

\(A\): \(n\times n\) matrix representing \(T\)

Roots of det\((A-\lambda I)\) have to be all real.

What about eigenvectors?

What about geometric multiplicity?

What about diagonalizability?

Real Spectral Theorem

\(V\): vector space over \(\mathbb{R}\)

\(T:V\to V\), an operator

The following are equivalent:

  1. \(T\) is self-adjoint
  2. \(V\) has an orthonormal basis of eigenvectors of \(T\)
  3. \(T\) is diagonal w.r.t. an orthonormal basis

Proof

Clearly, (2) implies (3) and vice versa

Proof of (3) implies (1)

Let \(D\) be the diagonal matrix representing \(T\) w.r.t. an orthonormal basis

Diagonal values of \(D\) are the real eigenvalues

So, conjugate-transpose of \(D\) is equal to \(D\), or \(T\): self-adjoint

Proof of real spectral theorem (continued)

Proof of (1) implies (3)

Let \(\lambda\) be a real eigenvalue of \(T\) with a real eigenvector \(v\).

Extend \(v\) to an orthonormal basis: \(\{v,u_1,\ldots,u_{n-1}\}\) (all real)

\(A=\begin{bmatrix} \lambda&a_{12}&\cdots&a_{1n}\\ 0 & & & \\ \vdots & & A_1 & \\ 0 & & & \end{bmatrix}\) (matrix of \(T\) w.r.t. above basis)

Since \(A=A^T\), we have the following:

  1. \(a_{12}=\cdots=a_{1n}=0\)

  2. \(A_1=A^T_1\): \((n-1)\times(n-1)\), self-adjoint

\(A_1\): represents a self-adjoint operator from \(\{u_1,\ldots,u_{n-1}\}\to\{u_1,\ldots,u_{n-1}\}\)

Repeat same argument with \(A_1\)

Finally, get a diagonal matrix for \(T\) w.r.t. an orthonormal basis for \(V\)

Matrices of self-adjoint operators on real spaces

\(A\): \(n\times n\) real, symmetric matrix (representing a self-adjoint operator w.r.t. standard basis)

There is a real, orthonormal basis \(\{e_1,\ldots,e_n\}\) s.t.

  1. \(e_i\) is an eigenvector of \(A\), or \(Ae_i=\lambda e_i\), \(\lambda_i\) real

  2. \(A=\lambda_1e_1\overline{e^T_1}+\cdots+\lambda_ne_n\overline{e^T_n}\)

Example

\(A=\begin{bmatrix} 14&-13&8\\ -13&14&8\\ 8&8&-7 \end{bmatrix}\)

\(e_1=\frac{1}{\sqrt{2}}(1,-1,0)\), \(e_2=\frac{1}{\sqrt{3}}(1,1,1)\), \(e_3=\frac{1}{\sqrt{6}}(1,1,-2)\)

\(\lambda_1=27\), \(\lambda_2=9\), \(\lambda_3=-15\)

\(A=\frac{27}{2}\begin{bmatrix}1\\-1\\0\end{bmatrix}\begin{bmatrix}1&-1&0\end{bmatrix} +\frac{9}{3}\begin{bmatrix}1\\1\\1\end{bmatrix}\begin{bmatrix}1&1&1\end{bmatrix} -\frac{15}{6}\begin{bmatrix}1\\1\\-2\end{bmatrix}\begin{bmatrix}1&1&-2\end{bmatrix}\)

Summary: Normal and self-adjoint operators

\(\{e_1,\ldots,e_n\}\): orthonormal basis

\(A=\lambda_1e_1\overline{e^T_1}+\cdots+\lambda_ne_n\overline{e^T_n}\)

Type \(e_i\) \(\lambda_i\)
Normal (complex) complex complex
Self-adjoint (complex) complex real
Self-adjoint (real) real real

Powers of \(A\)

\(A^k=\lambda^k_1e_1\overline{e^T_1}+\cdots+\lambda^k_ne_n\overline{e^T_n}\)

Order eigenvalues by magnitude \(\lvert\lambda_1\rvert \ge \cdots \ge \lvert\lambda_n\rvert\)

\(A^k\to\lambda^k_1e_1\overline{e^T_1}\) as \(k\to\infty\) (assume \(|\lambda_1|\) is the unique maximum)

Rank-\(r\) approximation of \(A\): \(\lambda_1e_1\overline{e^T_1}+\cdots+\lambda_re_r\overline{e^T_r}\)