Applied Linear Algebra

Andrew Thangaraj

Aug-Nov 2020

Course information

Syllabus

What is a scalar (or a number)?

  • Physical: represents the amount or quantity of something
  • Algebraic:
    • Defined through the algebraic operations that can be performed
    • Abstract: we do not really define what it is, but only what can be done with it
  • Scalars: something that we can add, subtract, multiply and divide
    • Set of scalars form a field
  • Examples
    • Rational numbers \(\mathbb{Q}\)
    • Real numbers \(\mathbb{R}\)
    • Complex numbers \(\mathbb{C}\)
  • In this course, we will consider real field \(\mathbb{R}\) and complex field \(\mathbb{C}\).

Quiz: what is a vector?

Sets and operations

  • Set \(S\)
    • Elements of the set are objects of our interest
  • Binary operation ‘+’ (addition) on elements of \(S\)
    • Given two elements \(a,b\in S\), assign an element of \(S\) as \(a+b\)
    • Commutative if \(a+b=b+a\)
  • Examples
    • Integers, rational numbers, real numbers, complex numbers with usual addition
    • Polynomials with polynomial addition
    • Functions with usual addition of functions
    • Grayscale images with pixelwise addition
    • Matrices with matrix addition
  • Identity element of addition: denoted 0
    • \(a+0=0+a=a\) for all \(a\)
  • Additive inverse of an element \(a\): denoted \(-a\)
    • \(a+(-a)=0\), or, in short, \(a-a=0\)

Vector space \(V\) over \(F\)

  • Field of scalars \(F\) and set of vectors \(V\)
    • Scalars: elements of \(F\)
    • Vectors: elements of \(V\)
  • Two operations
    • Vector addition: \(+\) (called plus) defined on \(V\)
    • Scalar multiplication: \(\cdot\) (called dot) defined on \(F\) and \(V\)
      • Given \(a\in F\) and \(v\in V\), assign a vector as \(a\cdot v\)
      • Often, \(\cdot\) is dropped and \(a\cdot v\) is denoted \(av\)
  • Requirements on operations
    • \(V\) with addition is an Abelian group (identity, inverse, associative, commutative)
    • Multiplicative Identity
      • \(1\): multiplicative identity of scalar field \(F\)
      • \(1v=v\)
    • Distributive properties
      • \(a(u+v) = au + av\)
      • \((a+b)v = av + bv\)
  • In simple terms…
    • Vectors can be scaled to get another vector
    • Two vectors can be added to get another vector

\(F^n\), where \(F=\mathbb{R}\) or \(\mathbb{C}\)

\[F^n=\{v=\begin{bmatrix}v_1\\ v_2\\ \vdots\\ v_n\end{bmatrix}: v_i\in F\}\]

  • Addition
    • \(u+v=\begin{bmatrix}u_1+v_1\\ u_2+v_2\\ \vdots\\ u_n+v_n\end{bmatrix}\)
  • Scalar multiplication
    • \(av=\begin{bmatrix}av_1\\ av_2\\ \vdots\\ av_n\end{bmatrix}\)
  • Exercise: check all requirements in definition

\(\mathbb{R}^n\) and \(\mathbb{C}^n\): most important of the examples

Polynomials

\[F_n[x]=\{v(x)=v_0+v_1x+v_2x^2+\cdots+v_nx^n: v_i\in F\}\]

  • Addition
    • \(u(x)+v(x)=(u_0+v_0)+(u_1+v_1)x+(u_2+v_2)x^2+\cdots+(u_n+v_n)x^n\)
  • Scalar multiplication
    • \(av(x)=av_0+av_1x+av_2x^2+\cdots+av_nx^n\)
  • Exercise: check all requirements in definition

You can think of a polynomial as a vector!

Matrices

\[F_{m,n}=\{A = \begin{bmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\ a_{21}&a_{22}&\cdots&a_{2n}\\ &\vdots&\vdots&\\ a_{m1}&a_{m2}&\cdots&a_{mn}\end{bmatrix}: a_{ij}\in F\}\]

  • \(A=[a_{ij}]\) - shorthand notation
  • Addition
    • \(A+B=[a_{ij}+b_{ij}]\)
  • Scalar multiplication
    • \(cA = [ca_{ij}]\)
  • Exercise: check all requirements in definition

You can think of a matrix as a vector! Image is a matrix.

Functions

\[\mathcal{F} = \{f:\mathbb{R}\to\mathbb{R}\}\]

  • Set of all functions from \(\mathbb{R}\) to \(\mathbb{R}\)
    • \(\sin x\), \(\tanh(x)\), \(x\), \(x^e\), \(e^x\) etc
  • Addition
    • \(f(x)+g(x)\)
  • Scalar multiplication
    • \(cf(x)\)
  • Exercise: check all requirements in definition

You can think of a function as a vector!

Vectors in Linear Algebra

When you hear “vector”…

  • Do not think only of magnitude and direction
  • Do not think only of a list of numbers

Think of vectors as things that you can add and scale.

Ideas from linear algebra apply to all things that can be thought of as a vector.