Dot product and length in \(\mathbb{C}^n\), Inner product and norm in \(V\) over \(F\)

Andrew Thangaraj

Aug-Nov 2020

Recap

  • Vector space \(V\) over a scalar field \(F\)
    • \(F\): real field \(\mathbb{R}\) or complex field \(\mathbb{C}\) in this course
  • \(m\times n\) matrix A represents a linear map \(T:F^n\to F^m\)
    • dim null \(T\) + dim range \(T\) = dim \(V\)
  • Linear equation: \(Ax=b\)
    • Solution (if it exists): \(u+\) null\((A)\)
  • Four fundamental subspaces of a matrix
    • Column space, row space, null space, left null space
  • Eigenvalue \(\lambda\) and Eigenvector \(v\): \(Tv=\lambda v\)
    • Distinct eigenvalues have independent eigenvectors
    • Basis of eigenvectors results in a diagonal matrix for \(T\)
    • Every linear map has an upper triangular matrix representation

Dot product and length in \(\mathbb{R}^2\)

\(\mathbb{R}^2\): Dot product of \(x=(x_1,x_2)\) and \(y=(y_1,y_2)\)

\(\langle x,y \rangle=x_1y_1+x_2y_2\in\mathbb{R}\)

  • Fix \(y\): linear in \(x\)

  • Related to angle between \(x\) and \(y\)

\(\mathbb{R}^2\): norm of \(x=(x_1,x_2)\)

\(\lVert x\rVert=\sqrt{\langle x,x\rangle}=\sqrt{x^2_1+x^2_2}\in\mathbb{R}^+\)

  • Nonlinear, but closely connected to dot product

  • Length or magnitude of \(x\) or distance from origin

Dot product and length in \(\mathbb{C}^n\)

\(\mathbb{C}^n\): Dot product of \(x=(x_1,\ldots,x_n)\) and \(y=(y_1,\ldots,y_n)\)

\(\langle x,y \rangle=x_1\overline{y_1}+\cdots+x_n\overline{y_n}\in\mathbb{C}\)

  • Fix \(y\): linear in \(x\)

  • Conjugation in the second argument

  • Generalize notion of angle to \(n\) dimensions

\(\mathbb{C}^n\): norm of \(x=(x_1,\ldots,x_n)\)

\(\lVert x\rVert=\sqrt{\langle x,x\rangle}=\sqrt{\lvert x_1\rvert^2+\cdots+\lvert x_n\rvert^2}\in\mathbb{R}^+\)

  • Nonlinear, but closely connected to dot product

  • Conjugation in the second argument is important

  • Length/magnitude in \(n\) dimensions

Inner product

\(V\): vector space over \(F=\mathbb{R}\) or \(\mathbb{C}\)

Inner product on \(V\): function mapping two vectors \(u,v\in V\) (in that order) to a scalar \(\langle u,v\rangle\in F\) satisfying the following properties.

  1. positivity: \(\langle v,v\rangle\ge0\)

  2. definiteness: \(\langle v,v\rangle=0\) if and only if \(v=0\)

  3. additivity in first argument: \(\langle u_1+u_2,v\rangle=\langle u_1,v\rangle+\langle u_2,v\rangle\)

  4. homogeneity in first argument: \(\langle \lambda u,v\rangle=\lambda\langle u,v\rangle\), \(\lambda\in F\)

  5. conjugate symmetry: \(\langle u,v\rangle=\overline{\langle v,u\rangle}\)

Examples: \(\mathbb{R}^n\) and \(\mathbb{C}^n\)

Dot product in \(\mathbb{R}^n\) and \(\mathbb{C}^n\)

Yes

\(\langle x,y\rangle=x_1y_1+2x_2y_2\) in \(\mathbb{R}^2\)

Yes

\(\langle x,y\rangle=x_1y_1-3x_1y_2-3x_2y_1+20x_2y_2\) in \(\mathbb{R}^2\)

Yes

\(\langle x,y\rangle=x_1y_1-3x_1y_2-3x_2y_1+9x_2y_2\) in \(\mathbb{R}^2\)

No

Examples: Functions form \([0,1]\) to \(\mathbb{R}\)

\[\langle f,g\rangle=\int_{0}^1 f(x)g(x)dx\]

Yes

\[\langle f,g\rangle=\int_{0}^1 f(x)g(x)e^{-x}dx\]

Yes

\[\langle f,g\rangle=\int_{0}^1 f(x)g(x-1/2)dx\]

No

Inner product space and properties

\(V\): vector space over \(F\) is called an inner product space if there is a valid inner product defined on \(V\).

Properties

  1. Fix \(u\in V\). Then, \(Tv=\langle v,u\rangle\) defines a linear map.

  2. \(\langle u,0\rangle=\langle 0,u\rangle=0\)

  3. \(\langle u,v_1+v_2\rangle=\langle u,v_1\rangle+\langle u,v_2\rangle\)

  4. \(\langle u,\lambda v\rangle=\overline{\lambda}\langle u,v\rangle\)

Orthogonality and orthogonal decomposition

\(V\): inner product space

\(u,v\) are orthogonal if \(\langle u,v\rangle=0\)

Orthogonal decomposition

For \(u,v\in V\) with \(v\ne0\), there exists \(c\) such that \(\langle u-cv,v\rangle=0\).

\(c=\langle u,v\rangle/\langle v,v\rangle\)

\(u = \dfrac{\langle u,v\rangle}{\langle v,v\rangle}v + w\)

  • \(cv\): parallel to \(v\)

  • \(w=u-cv\): orthogonal to \(v\)

Cauchy-Schwarz inequality

\(V\): inner product space and \(u,v\in V\). Then \(|\langle u,v\rangle|^2\le \langle u,u\rangle\langle v,v\rangle\)

Proof

  • If \(v=0\), done

  • If \(v\ne0\), orthogonal decomposition

\(u = \dfrac{\langle u,v\rangle}{\langle v,v\rangle}v + w\), and \(\langle v,w\rangle=0\)

\(\begin{align} \langle u,u\rangle&=\dfrac{|\langle u,v\rangle|^2}{\langle v,v\rangle^2}\langle v,v\rangle+\langle w,w\rangle\\ &\ge \dfrac{|\langle u,v\rangle|^2}{\langle v,v\rangle} \end{align}\)

Norm

\(V\): vector space over \(F=\mathbb{R}\) or \(\mathbb{C}\)

Norm on \(V\): function mapping vector \(v\in V\) to a scalar \(\lVert v\rVert\in \mathbb{R}\) satisfying the following properties.

  1. positivity: \(\lVert v\rVert\ge0\)

  2. definiteness: \(\lVert v\rVert=0\) if and only if \(v=0\)

  3. absolute scalability: \(\lVert \lambda v\rVert=\lvert \lambda\rvert \lVert v \rVert\)

  4. triangle inequality: \(\lVert u+v \rVert\le \lVert u \rVert + \lVert v \rVert\)

Examples: \(\mathbb{R}^n\) and \(\mathbb{C}^n\)

Euclidean or 2-norm or square norm

\[\lVert x\rVert=\sqrt{\lvert x_1\rvert^2+\cdots+\lvert x_n\rvert^2}\]

1-norm or Manhattan norm or taxicab norm

\[\lVert x\rVert=\lvert x_1\rvert+\cdots+\lvert x_n\rvert\]

\(\infty\)-norm or max norm

\[\lVert x\rVert=\max_{k=1,\ldots,n} \lvert x_k \rvert\]

\(p\)-norm, \(p\ge 1\)

\[\lVert x\rVert=\left(\lvert x_1\rvert^p+\cdots+\lvert x_n\rvert^p\right)^{1/p}\]

Norm from inner product

\(V\): inner product space. \(\lVert v\rVert=\sqrt{\langle v,v\rangle}\) is a norm.

Cauchy-Shwarz: \(\lvert\langle u,v\rangle\rvert\le \lVert u\rVert\lVert v\rVert\)

Proof for triangle inequality

\(\begin{align} \lVert u+v\lVert^2 &= \langle u+v,u+v\rangle\\ &= \langle u,u\rangle + \langle u,v\rangle + \langle v,u\rangle + \langle v,v\rangle\\ &= \langle u,u\rangle + 2 \text{Re}\langle u,v\rangle + \langle v,v\rangle\\ &\le \langle u,u\rangle + 2 \lvert\langle u,v\rangle\rvert + \langle v,v\rangle\\ &\le \lVert u\rVert^2+2\lVert u\rVert\lVert v\rVert+\lVert v\rVert^2\\ &=(\lVert u\rVert+\lVert v\rVert)^2 \end{align}\)

2-norm: from dot product

\(p\)-norm for \(p\ne2\): not from any inner product

Two geometric results

\(V\): inner product space and norm defined from inner product

Pythogorean law: if \(u,v\in V\) are orthogonal, \[\lVert u+v\rVert^2=\lVert u\rVert^2+\lVert v\rVert^2\]

Parallelogram law: if \(u,v\in V\), \[\lVert u+v\rVert^2+\lVert u-v\rVert^2=2(\lVert u\rVert^2+\lVert v\rVert^2)\]

Exercise: A norm is from an inner product if and only if it satisfies the parallelogram law.