Andrew Thangaraj
Aug-Nov 2020
\(T:V\to W\), linear map
Singular values of \(T\) are the eigenvalues of \(\sqrt{T^*T}\).
\(T^*T:V\to V\): (self-adjoint) positive operator.
Spectral theorem: \[T^*T\leftrightarrow \lambda_1e_1\overline{e^T_1}+\cdots+\lambda_ne_n\overline{e^T_n}\] Eigenvalues: \(\lambda_1\ge\cdots\ge\lambda_n\)
\(\{e_1,\ldots,e_n\}\): orthonormal basis, \(n=\) dim \(V\)
\(T^*T\) has a unique positive square root \(\sqrt{T^*T}\) \[\sqrt{T^*T}\leftrightarrow \sqrt{\lambda_1}e_1\overline{e^T_1}+\cdots+\sqrt{\lambda_n}e_n\overline{e^T_n}\]
Singular values of \(T\): \(\sqrt{\lambda_1}\ge\cdots\ge\sqrt{\lambda_n}\)
\(T:V\to W\), linear map
\(T^*T:V\to V\), self-adjoint
Right-singular vectors of \(T\) are an orthonormal eigenvector basis vectors of \(T^*T\)
Note: Right-singular vectors are vectors in \(V\)
\(TT^*:W\to W\), self-adjoint
Left-singular vectors of \(T\) are an orthonormal eigenvector basis vectors of \(TT^*\)
Note: Left-singular vectors are vectors in \(W\)
\(A=\begin{bmatrix}1&2\\3&4\end{bmatrix}\) (standard basis)
Eigenvalues: \(5.372\), \(-0.372\); Eigenvectors: \((0.415,0.909)\), \((0.824,-0.566)\)
\(\begin{bmatrix}1&2\\3&4\end{bmatrix}\begin{bmatrix}0.415\\0.909\end{bmatrix}=5.372\begin{bmatrix}0.415\\0.909\end{bmatrix},\quad\begin{bmatrix}1&2\\3&4\end{bmatrix}\begin{bmatrix}0.824\\-0.566\end{bmatrix}=-0.372\begin{bmatrix}0.824\\-0.566\end{bmatrix}\)
\(A^TA=\begin{bmatrix}10&14\\14&20\end{bmatrix}\), \(AA^T=\begin{bmatrix}5&11\\11&25\end{bmatrix}\)
Eigenvalues of \(A^TA\) and \(AA^T\): \(29.866\), \(0.134\)
Singular values of \(A\): \(\sigma_1=5.465\), \(\sigma_2=0.366\)
Right-singular vectors of \(A\): \(e_1=(0.576,0.817)\), \(e_2=(0.817,-0.576)\)
Left-singular vectors of \(A\): \(f_1=(0.404,0.914)\), \(f_2=(-0.914,0.404)\)
\(\begin{bmatrix}1&2\\3&4\end{bmatrix}\begin{bmatrix}0.576\\0.817\end{bmatrix}=5.465\begin{bmatrix}0.404\\0.914\end{bmatrix},\quad\begin{bmatrix}1&2\\3&4\end{bmatrix}\begin{bmatrix}0.817\\-0.576\end{bmatrix}=0.366\begin{bmatrix}-0.914\\0.404\end{bmatrix}\)
Observation: \(Ae_1=\sigma_1f_1\) and \(Ae_2=\sigma_2f_2\) (This is not an accident)
\(A=\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}\) (eigenvalues undefined)
\(A^TA=\begin{bmatrix}17&22&27\\22&29&36\\27&36&45\end{bmatrix}\), \(AA^T=\begin{bmatrix}14&32\\32&77\end{bmatrix}\)
Eigenvalues of \(A^TA\): \(90.403\), \(0.597\), \(0\)
Eigenvalues of \(AA^T\): \(90.403\), \(0.597\)
Singular values of \(A\): \(\sigma_1=9.508\), \(\sigma_2=0.773\), \(\sigma_3=0\)
Right-singular vectors of \(A\): \(e_1=(0.429,0.566,0.704)\), \(e_2=(0.805,0.112,-0.581)\), \(e_3=(0.408,-0.816,0.408)\)
Left-singular vectors of \(A\): \(f_1=(0.386,0.922)\), \(f_2=(-0.922,0.386)\)
\(\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}\begin{bmatrix}0.429\\0.566\\0.704\end{bmatrix}=9.508\begin{bmatrix}0.386\\0.922\end{bmatrix},\quad\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}\begin{bmatrix}0.805\\0.112\\-0.581\end{bmatrix}=0.773\begin{bmatrix}-0.922\\0.386\end{bmatrix}\)
\(Ae_1=\sigma_1f_1\), \(Ae_2=\sigma_2f_2\) and \(Ae_3=0\)
\(T:V\to W\), \(T^*T:V\to V\), \(\sqrt{T^*T}:V\to V\), \(TT^*:W\to W\)
null \(T=\) \((\)range \(T^*)^{\perp}\)
null \(T^*=\) \((\)range \(T)^{\perp}\)
dim range \(T=\) dim range \(T^*\)
null \(T^*T=\) null \(T\)
range \(T^*T=\) \((\)null \(T^*T)^{\perp}=\) \((\)null \(T)^{\perp}=\) range \(T^*\)
dim range \(T^*T=\) dim range \(T^*=\) dim range \(T\)
null \(\sqrt{T^*T}=\) null \(T^*T=\) null \(T\)
range \(\sqrt{T^*T}=\) \((\)null \(\sqrt{T^*T})^{\perp}=\) \((\)null \(T)^{\perp}\)
dim range \(\sqrt{T^*T}=\) dim range \(T\)
null \(TT^*=\) null \(T^*=\) \((\)range \(TT^*)^{\perp}\)
range \(TT^*=\) \((\)null \(T^*)^{\perp}=\) range \(T\)