Linear Functionals, Orthogonal Complements

Andrew Thangaraj

Aug-Nov 2020

Recap

Vector space \(V\) over a scalar field \(F\)
- \(F\): real field \(\mathbb{R}\) or complex field \(\mathbb{C}\) in this course
\(m\times n\) matrix A represents a linear map \(T:F^n\to F^m\)
- dim null \(T\) + dim range \(T\) = dim \(V\)
Linear equation: \(Ax=b\)
- Solution (if it exists): \(u+\) null\((A)\)
Four fundamental subspaces of a matrix
- Column space, row space, null space, left null space
Eigenvalue \(\lambda\) and Eigenvector \(v\): \(Tv=\lambda v\)
- Upper triangular matrix for a linear map
- Some linear maps are diagonalizable
Inner products, norms
- Orthogonality and orthonormal basis
- Upper triangular matrix for a linear map over an orthonormal basis

Linear functionals

\(V\): vector space

A linear functional is a linear map from \(V\) to \(F\).

Examples

\(\phi:\mathbb{R}^3\to\mathbb{R}\), \(\phi(x_1,x_2,x_3)=2x_1-5x_2+7x_3\)
\(\phi:F^n\to F\), \(\phi(x_1,\ldots,x_n)=c_1x_1+\cdots+c_nx_n\)

All linear functionals from \(F^n\) to \(F\) will be like above
\(P_2(\mathbb{R})\): polynomials of deg \(\le 2\), \(\phi:P_2(\mathbb{R})\to\mathbb{R}\) defined as \[\phi(p(x))=\int_0^1 p(x)\cos(\pi x)dx\]
- Is there a simpler description?

Linear functionals in inner product spaces

\(V\): inner product space

Example of linear functional: Fix \(u\in V\). \(\phi(v)=\langle v,u\rangle\)

Can there be any other type?

Riesz representation theorem: \(V\) is a finite-dimensional inner product space and \(\phi\) is a linear functional on \(V\). Then, there exists a unique vector \(u\) such that \(\phi(v)=\langle v,u\rangle\).

Proof

\(e_1,\cdots,e_n\): orthonormal basis for \(V\)

\[\begin{align} \phi(v)&=\phi(\langle v,e_1\rangle e_1+\cdots+\langle v,e_n\rangle e_n)\\ &=\langle v,e_1\rangle \phi(e_1) + \cdots + \langle v,e_n\rangle \phi(e_n)\\ &=\langle v,\overline{\phi(e_1)}e_1+\cdots+\overline{\phi(e_n)}e_n\rangle. \end{align}\]

So, \(u=\overline{\phi(e_1)}e_1+\cdots+\overline{\phi(e_n)}e_n\) is such that \(\phi(v)=\langle v,u\rangle\).

Example

\(P_2(\mathbb{R})\): polynomials of deg \(\le 2\), Inner product: \(\langle p,q\rangle=\int\limits_0^1 p(x)q(x)dx\)

Linear functional \(\phi:P_2(\mathbb{R})\to\mathbb{R}\) defined as \[\phi(p(x))=\int_0^1 p(x)\cos(\pi x)dx\]

Riesz: There exists \(q(x)=q_0+q_1x+q_2x^2\) such that \[\phi(x)=\int_0^1 p(x)\cos(\pi x)dx=\int_0^1 p(x)q(x)dx\]

Find orthonormal basis \(e_0(x),e_1(x),e_2(x)\)
- Gram-Schmidt on \(1,x,x^2\).
\(q(x)=\phi(e_0(x))e_0(x)+\phi(e_1(x))e_1(x)+\phi(e_2(x))e_2(x)\)

Orthogonal complements

\(U\subseteq V\), subset

Orthogonal complement of \(U\), denoted \(U^{\perp}\), is the set of vectors orthogonal to every vector in \(U\). \[U^{\perp}=\{v\in V: \langle v,u\rangle=0\text{ for every }u\in U\}\]

Examples: \(\mathbb{R}^2\)

\(U=0\)
\(U=(x_1,y_1)\)
\(U\): line through origin
\(U=\{(x_1,y_1),(x_2,y_2)\}\)
\(U=V\)

Basic properties: \(U\) is a subset of \(V\)

\(U^{\perp}\) is a subspace of \(V\)
\(\{0\}^{\perp}=V\), \(V^{\perp}=\{0\}\)
\(U\cap U^{\perp}=\emptyset\) or \(\{0\}\)
if \(U\subseteq W\), \(W^{\perp}\subseteq U^{\perp}\)

Examples: \(U\) is a subspace of \(\mathbb{R}^n\) or \(\mathbb{C}^n\)

\(U=\text{span}\{(1,2,3,4),(3,4,5,6)\}\)

\(U=\) rowspace\((A)\), where \(A=\begin{bmatrix} 1&2&3&4\\ 3&4&5&6 \end{bmatrix}\)

\(\begin{align} U^{\perp}&=\{v\in \mathbb{R}^4:\langle v,(1,2,3,4) \rangle=0, \langle v,(3,4,5,6) \rangle=0\}\\ &=\{v\in\mathbb{R}^4:Av=0\} \end{align}\)

\(U^{\perp}=\) null\((A)\)

=======================================================

\(U\) is a subspace of \(\mathbb{C}^n\), \(U=\) rowspace\((A)\)

\(U^{\perp}=\) null\((\overline{A})\), \(\overline{A}\): element-wise conjugate of \(A\)

Four fundamental subspaces of \(m\times n\) real matrix

\(\langle x,y \rangle=x_1y_1+\cdots+x_ny_n\)

\(A\): \(m\times n\) matrix

Result 1

null\((A)=\) rowspace\((A)^{\perp}\)

Result 2

range\((A)=\) left-null\((A)^{\perp}\)

Orthogonal complement of subspaces

if \(U\): finite-dimensional subspace of \(V\), then \(V=U\oplus U^{\perp}\)

Proof

\(e_1,\ldots,e_m\): orthonormal basis for \(U\)

Any \(v\in V\) can be written as \[v=(\langle v,e_1\rangle e_1+\cdots+\langle v,e_m\rangle e_m)+(v-\langle v,e_1\rangle e_1-\cdots-\langle v,e_m\rangle e_m)\] First term is in \(U\), and second term in \(U^{\perp}\) (why?)

Since \(U \cap U^{\perp}=\{0\}\), \(V=U\oplus U^{\perp}\).

Corollary (\(V\): finite-dimensional)

dim \(U^{\perp}=\) dim \(V-\) dim \(U\)

\(U\): subspace of a finite-dimensional \(V\)

Complement of complement

\(\left(U^{\perp}\right)^{\perp}=U\)

Orthonormal basis for \(U^{\perp}\)

Find an orthonormal basis for \(U\): \(e_1,\ldots,e_m\)
Extend to orthonormal basis of \(V\): \(e_1,\ldots,e_m,e_{m+1},\ldots,e_n\)
Orthonormal basis of \(U^{\perp}\): \(e_{m+1},\ldots,e_n\)

To specify \(U\)

Give a basis for \(U\)

Give a basis for \(U^{\perp}\)

Sum and intersection of subspaces

\(U\): basis \(u_1,\ldots,u_k\), \(W\): basis \(w_1,\ldots,w_l\)

sum of \(U\) and \(W\)

\(U+W\): spanning set \(u_1,\ldots,u_k,w_1,\ldots,w_l\)

Reduce by elementary row operations

intersection of \(U\) and \(W\)

\((U+W)^{\perp}=U^{\perp}\cap W^{\perp}\)

Proof

If \(v\in(U+W)^{\perp}\), \(v\in U^{\perp}\) and \(v\in W^{\perp}\)

\(U^{\perp}=\{v\in V: \langle v,u_i \rangle=0, i=1,\ldots,k\}\)

\(W^{\perp}=\{v\in V: \langle v,w_j \rangle=0, j=1,\ldots,l\}\)

\(U^{\perp}\cap W^{\perp}=\{v\in V: \langle v,u_i \rangle=0, i=1,\ldots,k\text{ and }\langle v,w_j \rangle=0, j=1,\ldots,l\}\)

Corollary: \(U\cap W=(U^{\perp}+W^{\perp})^{\perp}\)

Example

\(U=\) rowspace\(\begin{bmatrix} 1&2&3&4\\ 3&4&5&6 \end{bmatrix}\), \(W=\) rowspace\(\begin{bmatrix} 1&1&1&1\\ 1&-1&1&-1 \end{bmatrix}\)

\(U=\) null\(\begin{bmatrix} 1&-2&1&0\\ 2&-3&0&1 \end{bmatrix}\), \(W=\) null\(\begin{bmatrix} -1&0&1&0\\ 0&-1&0&1 \end{bmatrix}\)

\(U\cap W=\) null\(\begin{bmatrix} 1&0&-1&0\\ 0&1&0&-1\\ 1&-2&1&0\\ 2&-3&0&1 \end{bmatrix}=\) null\(\begin{bmatrix} 1&0&0&-1\\ 0&1&0&-1\\ 0&0&1&-1 \end{bmatrix}\)

\(U\cap W=\) rowspace\(\begin{bmatrix}1&1&1&1\end{bmatrix}\)