Applied Linear Algebra
Andrew Thangaraj
Aug-Nov 2020
What is a scalar (or a number)?
- Physical: represents the amount or quantity of something
- Algebraic:
- Defined through the algebraic operations that can be performed
- Abstract: we do not really define what it is, but only what can be done with it
- Scalars: something that we can add, subtract, multiply and divide
- Set of scalars form a field
- Examples
- Rational numbers \(\mathbb{Q}\)
- Real numbers \(\mathbb{R}\)
- Complex numbers \(\mathbb{C}\)
- In this course, we will consider real field \(\mathbb{R}\) and complex field \(\mathbb{C}\).
Sets and operations
- Set \(S\)
- Elements of the set are objects of our interest
- Binary operation ‘+’ (addition) on elements of \(S\)
- Given two elements \(a,b\in S\), assign an element of \(S\) as \(a+b\)
- Commutative if \(a+b=b+a\)
- Examples
- Integers, rational numbers, real numbers, complex numbers with usual addition
- Polynomials with polynomial addition
- Functions with usual addition of functions
- Grayscale images with pixelwise addition
- Matrices with matrix addition
- Identity element of addition: denoted 0
- \(a+0=0+a=a\) for all \(a\)
- Additive inverse of an element \(a\): denoted \(-a\)
- \(a+(-a)=0\), or, in short, \(a-a=0\)
Vector space \(V\) over \(F\)
- Field of scalars \(F\) and set of vectors \(V\)
- Scalars: elements of \(F\)
- Vectors: elements of \(V\)
- Two operations
- Vector addition: \(+\) (called plus) defined on \(V\)
- Scalar multiplication: \(\cdot\) (called dot) defined on \(F\) and \(V\)
- Given \(a\in F\) and \(v\in V\), assign a vector as \(a\cdot v\)
- Often, \(\cdot\) is dropped and \(a\cdot v\) is denoted \(av\)
- Requirements on operations
- \(V\) with addition is an Abelian group (identity, inverse, associative, commutative)
- Multiplicative Identity
- \(1\): multiplicative identity of scalar field \(F\)
- \(1v=v\)
- Distributive properties
- \(a(u+v) = au + av\)
- \((a+b)v = av + bv\)
- In simple terms…
- Vectors can be scaled to get another vector
- Two vectors can be added to get another vector
\(F^n\), where \(F=\mathbb{R}\) or \(\mathbb{C}\)
\[F^n=\{v=\begin{bmatrix}v_1\\ v_2\\ \vdots\\ v_n\end{bmatrix}: v_i\in F\}\]
- Addition
- \(u+v=\begin{bmatrix}u_1+v_1\\ u_2+v_2\\ \vdots\\ u_n+v_n\end{bmatrix}\)
- Scalar multiplication
- \(av=\begin{bmatrix}av_1\\ av_2\\ \vdots\\ av_n\end{bmatrix}\)
- Exercise: check all requirements in definition
\(\mathbb{R}^n\) and \(\mathbb{C}^n\): most important of the examples
Polynomials
\[F_n[x]=\{v(x)=v_0+v_1x+v_2x^2+\cdots+v_nx^n: v_i\in F\}\]
- Addition
- \(u(x)+v(x)=(u_0+v_0)+(u_1+v_1)x+(u_2+v_2)x^2+\cdots+(u_n+v_n)x^n\)
- Scalar multiplication
- \(av(x)=av_0+av_1x+av_2x^2+\cdots+av_nx^n\)
- Exercise: check all requirements in definition
You can think of a polynomial as a vector!
Matrices
\[F_{m,n}=\{A = \begin{bmatrix}a_{11}&a_{12}&\cdots&a_{1n}\\ a_{21}&a_{22}&\cdots&a_{2n}\\ &\vdots&\vdots&\\ a_{m1}&a_{m2}&\cdots&a_{mn}\end{bmatrix}: a_{ij}\in F\}\]
- \(A=[a_{ij}]\) - shorthand notation
- Addition
- Scalar multiplication
- Exercise: check all requirements in definition
You can think of a matrix as a vector! Image is a matrix.
Functions
\[\mathcal{F} = \{f:\mathbb{R}\to\mathbb{R}\}\]
- Set of all functions from \(\mathbb{R}\) to \(\mathbb{R}\)
- \(\sin x\), \(\tanh(x)\), \(x\), \(x^e\), \(e^x\) etc
- Addition
- Scalar multiplication
- Exercise: check all requirements in definition
You can think of a function as a vector!
Vectors in Linear Algebra
When you hear “vector”…
- Do not think only of magnitude and direction
- Do not think only of a list of numbers
Think of vectors as things that you can add and scale.
Ideas from linear algebra apply to all things that can be thought of as a vector.