Andrew Thangaraj
Aug-Nov 2020
\(U\): subspace of \(V\)
Is there a linear operator \(T\) s.t. range\((T)=U\)?
Yes… there are many!
Basis for \(V\): \(\{v_1,\ldots,v_n\}\)
Define \(T\) as mapping each \(v_i\) to some \(u_i\in U\)
Othogonal projection: linear operator taking \(v\in V\) into \(U\) in a specific way
\(U\): subspace of \(V\)
\(U^{\perp}\): orthogonal complement of \(U\)
\[U^{\perp}=\{v\in V: \langle v,u\rangle=0\text{ for all }u\in U\}\]
\[V=U\oplus U^{\perp}\]
How to find \(U^{\perp}\)?
Orthonormal basis for \(U\): \(\{e_1,\ldots,e_m\}\)
Extend to orthonormal basis for \(V\): \(\{e_1,\ldots,e_m,e_{m+1},\ldots,e_n\}\)
Orthonormal basis for \(U^{\perp}\): \(\{e_{m+1},\ldots,e_n\}\)
\(v=\langle v,e_1\rangle e_1+\cdots+\langle v,e_m\rangle e_m+\langle v,e_{m+1}\rangle e_{m+1}+\cdots+\langle v,e_n\rangle e_n\)
\(u=\langle v,e_1\rangle e_1+\cdots+\langle v,e_m\rangle e_m\in U\)
\(u^{\perp}=\langle v,e_{m+1}\rangle e_{m+1}+\cdots+\langle v,e_n\rangle e_n\)
\(v=u+u^{\perp}\)
\(U=\) span\(\{(1,0,0,0),(0,1,0,0)\}\)
Orthonormal basis extension
\(\{(1,0,0,0),(0,1,0,0),(0,0,1,0),(0,0,0,1)\}\)
\(U^{\perp}=\) span\(\{(0,0,1,0),(0,0,0,1)\}\)
\(v=(1,0,1,1)=u+u^{\perp}\), where \(u=(1,0,0,0)\), \(u^{\perp}=(0,0,1,1)\)
\(U=\) span\(\{(1/\sqrt{2},1/\sqrt{2},0,0),(1/\sqrt{2},-1/\sqrt{2},0,0)\}\)
Orthonormal basis extension
\(\{(1/\sqrt{2},1/\sqrt{2},0,0),(1/\sqrt{2},-1/\sqrt{2},0,0),(0,0,1/\sqrt{2},1/\sqrt{2}),(0,0,1/\sqrt{2},-1/\sqrt{2})\}^1\)
\(U^{\perp}=\{(0,0,1/\sqrt{2},1/\sqrt{2}),(0,0,1/\sqrt{2},-1/\sqrt{2})\}\)
\(v=(1,0,1,1)^1=u+u^{\perp}\), where \(u=(1,0,0,0)^1\), \(u^{\perp}=(0,0,1,1)^1\)
\(U\): finite-dimensional subspace of \(V\)
Orthogonal projection operator \(P_U\) maps \(v=u+u^{\perp}\), where \(u\in U\), \(u^{\perp}\in U^{\perp}\), to \(u\).
Example
\(x\in V\), \(x\ne0\), \(U=\) span\((x)\)
Orthonormal basis for \(U\): \(\dfrac{x}{\lVert x\rVert}\)
\[P_Uv=\frac{\langle v,x\rangle}{\lVert x\rVert^2}x\]
\(x,y\in V\), linearly independent, and \(U=\) span\((x,y)\)
Orthonormal basis for \(U\): \(e_1=\dfrac{x}{\lVert x\rVert}\), \(e_2=\dfrac{y-\langle y,e_1\rangle e_1}{\lVert y-\langle y,e_1\rangle e_1\rVert}\)
\[P_Uv=\langle v,e_1\rangle e_1+\langle v,e_2\rangle e_2\]
\(V=\mathbb{R}^n\), dot product, standard basis
\(U=\) span\(\{(x_1,\ldots,x_n)\}\)
\[P_Uv=\frac{\langle v,x\rangle}{\lVert x\rVert^2}x = \frac{1}{\lVert x\rVert^2}x(x^Tv)\]
\(P_U\leftrightarrow \dfrac{1}{\lVert x\rVert^2}xx^T\)
\(U=\) span\(\{e_1=(e_{11},\ldots,e_{1n}),\ldots,e_m=(e_{m1},\ldots,e_{mn})\}\)
\(e_1,\ldots,e_m\): orthonormal
\(\begin{align} P_Uv&=\langle v,e_1\rangle e_1+\cdots+\langle v,e_m\rangle e_m\\ &=(e_1e^T_1+\ldots+e_me^T_m)v \end{align}\)
\(P_U\leftrightarrow e_1e^T_1+\ldots+e_me^T_m\)
\(U\): finite-dimensional subspace of \(V\), \(v\in V\)
\(P_Uu=u\) for \(u\in U\)
\(P_Uw=0\) for \(w\in U^{\perp}\)
range \(P_U=U\)
null \(P_U=U^{\perp}\)
\(v-P_Uv\in U^{\perp}\)
\(P^2_U=P_U\)
\(\lVert P_Uv\rVert \le \lVert v\rVert\)