Linear Combinations, Span, Subspaces, Linear Dependence and Independence

Andrew Thangaraj

Aug-Nov 2020

Recap

  • Vector space \(V\) over a scalar field \(F\)
    • \(F\): real field \(\mathbb{R}\) or complex field \(\mathbb{C}\) in this course
    • Operations
      • vector addition: \(u+v\) for \(u,v\in V\)
      • scalar multiplication: \(av\) for \(a\in F\) and \(v\in V\)
    • Requirements: addition is Abelian, \(1v=v\), distributive properties
  • Algebraic/abstract notion of a vector
    • Vector is defined through its operations
    • Connection to physical nature is not emphasized

Linear Combinations

  • Vectors \(v_1,v_2,\ldots\in V\), Scalars \(a_1,a_2,\ldots\in F\)

Linear combination: \(a_1v_1+a_2v_2+\cdots\)

  • Examples: \(V=\mathbb{R}^3\)

    • \(a\ \begin{bmatrix}2\\ -1\\ 1\end{bmatrix}=\begin{bmatrix}2a\\ -a\\ a\end{bmatrix}\)

    • \(2\ \begin{bmatrix}5\\ 3\\ -4\end{bmatrix}+3.5\ \begin{bmatrix}2\\ -1\\ -2\end{bmatrix} = \begin{bmatrix}17\\ 2.5\\ -15\end{bmatrix}\)

    • \(4\ \begin{bmatrix}-2\\ 1\\ 6\end{bmatrix}-9\ \begin{bmatrix}5\\ -1\\ -3\end{bmatrix}+3\ \begin{bmatrix}1\\ 2\\ -2\end{bmatrix} = \begin{bmatrix}-50\\ 19\\ 45\end{bmatrix}\)

Linear combinations of a handful of vectors generate infinitely many vectors

Linear combinations are the essence of linear algebra

Span

Vectors \(v_1,v_2,\ldots,v_m\in V\)

span\((v_1,v_2,\ldots,v_m)=\{a_1v_1+a_2v_2+\cdots+a_mv_m:a_i\in F\}\)

  • Examples: \(V=\mathbb{R}^2\)

    1. \(v=\begin{bmatrix}1\\2\end{bmatrix}\), span\((v)=\{\begin{bmatrix}a\\2a\end{bmatrix}:a\in\mathbb{R}\}\)

      • Is \(\begin{bmatrix}3\\6\end{bmatrix}\) in the span? Is \(\begin{bmatrix}2\\10\end{bmatrix}\) in the span?

    2. \(v_1=\begin{bmatrix}1\\2\end{bmatrix}\), \(v_2=\begin{bmatrix}2\\5\end{bmatrix}\), span\((v_1,v_2)=\{\begin{bmatrix}a+2b\\2a+5b\end{bmatrix}:a,b\in\mathbb{R}\}\)

      • Is \(\begin{bmatrix}3\\6\end{bmatrix}\) in the span? Is \(\begin{bmatrix}2\\5\end{bmatrix}\) in the span?

      • Is \(\begin{bmatrix}2\\10\end{bmatrix}\) in the span?

Span (continued)

  • Examples: \(V=\mathbb{R}^3\)

    1. \(v_1=\begin{bmatrix}1\\2\\3\end{bmatrix}\), \(v_2=\begin{bmatrix}2\\3\\4\end{bmatrix}\), span\((v_1,v_2)=\{\begin{bmatrix}a+2b\\2a+3b\\3a+4b\end{bmatrix}:a,b\in\mathbb{R}\}\)

      • Is \(\begin{bmatrix}2\\8\\10\end{bmatrix}\) in the span?

    2. \(v_1=\begin{bmatrix}1\\0\\0\end{bmatrix}\), \(v_2=\begin{bmatrix}0\\1\\0\end{bmatrix}\), \(v_3=\begin{bmatrix}0\\0\\1\end{bmatrix}\). Is \(\begin{bmatrix}x\\y\\z\end{bmatrix}\) in the span?

  • Example: \(V=\mathbb{R}^{1000}\)

    • 100 vectors given: \(v_1\), \(\ldots\), \(v_{100}\)

    • Ask if 101-st vector is in the span?

Subspaces

\(U\subseteq V\) is a subspace if and only if \(U\) is closed under vector addition and scalar multiplication.

  • Equivalent to saying closed under linear combinations
    • \(u_1,u_2\in U\) implies \(au_1+bu_2\in U\) for any \(a,b\in F\).
  • Why study subspaces?
    • Divide and conquer: understand vector space by understanding subspaces
  • Examples
    • In \(\mathbb{R}^2\) and \(\mathbb{R}^3\), lines and planes through origin
    • \(\{(x,x,y)\in F^3: x,y\in F\}\)
    • \(\{(x,y,z)\in F^3: x+y+z=0\}\)

Exercise: span\((v_1,v_2,\ldots,v_m)\) is the smallest subspace containing \(v_i\).

Linear Dependence and Independence

Vectors \(v_1,v_2,\ldots,v_m\in V\) are said to be linearly dependent if there exist scalars \(a_1,a_2,\ldots,a_m\in F\), not all zero, such that \(a_1v_1+a_2v_2+\cdots+a_mv_m=0\).

  • Linearly dependent if there is a non-trivial linear combination that can result in the zero vector.

Vectors \(v_1,v_2,\ldots,v_m\in V\) are said to be linearly independent if \(a_1v_1+a_2v_2+\cdots+a_mv_m=0\) implies \(a_i=0\) for \(i=1,2,\ldots,m\).

  • Linearly independent if only the trivial linear combination results in the zero vector.

Linear dependence of two vectors

  • Examples: In \(V=\mathbb{R}^2\), are the following linear dependent or independent?

    1. \(v_1=\begin{bmatrix}1\\2\end{bmatrix}\), \(v_2=\begin{bmatrix}3\\6\end{bmatrix}\)

    2. \(v_1=\begin{bmatrix}1\\2\end{bmatrix}\), \(v_2=\begin{bmatrix}2\\5\end{bmatrix}\)

  • Examples: In \(V=\mathbb{R}^3\), are the following linear dependent or independent?

    1. \(v_1=\begin{bmatrix}1\\2\\5\end{bmatrix}\), \(v_2=\begin{bmatrix}3\\6\\15\end{bmatrix}\)

    2. \(v_1=\begin{bmatrix}1\\2\\5\end{bmatrix}\), \(v_2=\begin{bmatrix}3\\6\\10\end{bmatrix}\)

Exercise: Prove that two vectors are linearly dependent if and only if one is a multiple of the other.

Linear dependence of 3 vectors

  • Examples: In \(V=\mathbb{R}^2\), are the following linear dependent or independent?

    1. \(v_1=\begin{bmatrix}1\\0\end{bmatrix}\), \(v_2=\begin{bmatrix}0\\1\end{bmatrix}\), \(v_3=\begin{bmatrix}3\\17\end{bmatrix}\)

    2. \(v_1=\begin{bmatrix}1\\2\end{bmatrix}\), \(v_2=\begin{bmatrix}2\\5\end{bmatrix}\), \(v_3=\begin{bmatrix}3\\17\end{bmatrix}\)

  • Exercise: Prove that any 3 vectors in \(\mathbb{R}^2\) are linearly dependent.

  • Examples: In \(V=\mathbb{R}^3\), are the following linear dependent or independent?

    1. \(v_1=\begin{bmatrix}1\\0\\0\end{bmatrix}\), \(v_2=\begin{bmatrix}0\\1\\0\end{bmatrix}\), \(v_3=\begin{bmatrix}a\\b\\c\end{bmatrix}\)

    2. \(v_1=\begin{bmatrix}1\\2\\5\end{bmatrix}\), \(v_2=\begin{bmatrix}3\\6\\7\end{bmatrix}\), \(v_3=\begin{bmatrix}a\\b\\c\end{bmatrix}\)

How to establish linear (in)dependence of vectors?

  • Example: \(V=\mathbb{R}^{1000}\)
    • 100 vectors given: \(v_1\), \(\ldots\), \(v_{100}\)
    • Are they linearly independent?
    • Equivalent question: Is \(v_{100}\) in span\((v_1,\ldots,v_{99})\)?
  • Special case
    • \(v_i\) has 1 in \(i\)-th position and zero in all other positions from 1 to 100
    • Are they linearly independent? Yes.
  • Result: The question of linear dependence of any set of vectors can be manipulated to make it look like the special case above.
    • How? Gaussian elimination

Quiz