Math-Intro to Linear Algebra

Introduction to matrices-

Matrices- A matrix is a rectangular array with some elements.

A common use for matrices is to represent the coefficients of systems of equations. The standard ways to solve system of equations involve using a substitution method or an elimination method by adding and subtracting equations from one another. For example:

x ₁ - 2 x ₂ + x ₃ = 0

2 x ₂ - 8 x ₃ = 8

-4 x ₁ + 5 x ₂ + 9 x ₃ = -9

In a substitution method you would normally do something along the lines of using equation 2 to set 2 x ₂ = 8 + 8 x ₃  to get x ₂ = 4 + 4 x ₃ and substitute into the other equations.

In an elimination method you can add two equations together in order to get rid of terms. For example adding equation one and two will give you x ₁ - 7 x ₃ = 8, and continue doing that to solve equations.

This can be summed up in a more simple fashion by using matrices, which contain the coefficients of the system of equations.
[ 1 -2 1 0 ]
| 0 2 -8 8 |
[ -4 5 9 -9 ]
Then use row operations in order to create a diagonalized matrix. There are a few basic row operations that can be used in diagonalizing a matrix such as this.

Scaling-Multiply all the entries in a row by a nonzero constant

Interchange-Interchange two rows

Replacement-Replace one row by the sum of itself and a multiple of another row.

Diagonalization means to reduce the matrix to the identity matrix, which contains only ones on the diagonal. The 4th column in the previous matrix represents solutions so the matrix can be rewritten as:
[ 1 -2 1 | 0 ]
| 0 2 -8 | 8 |
[ -4 5 9 | -9 ]
where | represents the separation between the matrices. The matrix in reduced form can also be called an echelon matrix or in reduced echelon form when the diagonals are 0, and only the solutions remain. Should a row solve to 0 0 0 0, this means that the variable this row is supposed to solve for is free and can take any value.

So to solve this we can perform operations such as

[ 1 -2 1 0 ]
| 0 2 -8 8 |
[ -4 5 9 -9 ]

[ 1 0 -7 8 ]
| 0 2 -8 8 |
[-4 5 9 -9 ]

[ 1 0 -7 8 ]
| 0 2 -8 8 |
[ 0 5 -21 23 ]

[ 1 0 -7 8 ]
| 0 2 -8 8 |
[ 0 5 -21 23 ]

[ 1 0 -7 8 ]
| 0 2 -8 8 |
[ 0 0 1 3 ]

[ 1 0 0 29 ]
| 0 2 -8 8 |
[ 0 0 1 3 ]

[ 1 0 0 29]
| 0 2 0 32|
[ 0 0 1 3 ]

[ 1 0 0 29]
| 0 1 0 16|
[ 0 0 1 3 ]

This result means that x ₁ = 29, x ₂ = 16, and x ₃ = 3.

Cross-product definition-

In matrix form the cross product of two 3 element vectors is defined as:

(a b c) x (x y z)

[ i j k ]
| a b c |
[ x y z ]

(bz - cy)i -(az - cx)j + (ay - bx)k

Where i j k are the unit vectors that define the axis in which both vectors are defined.

Least Squares and Data fitting
In order to find the least square solution, or a regression fit of a set of data, we calculate  
A^T Ax = A^T b 
However, if the ker(A) = {0} we can also get a unique least squares solution of 
x = (A^TA)^-1 A^Tb


Example:
A = [1 1]
        |1 2|
        [1 3]
b = [0]
       |0|
       [6]


So we can compute A^T as
[1 1 1 ]
[1 2 3]
so A^TA is
[ 3 6  ]
[ 6 14]
Then we take the inverse
[2.3 -1]
[-1   .5]
Next we solve for A^Tb
[ 6 ]
[18]
multiple the above two together to get
x = [ -4]
       [ 3 ]


Inner products
An inner product is a linear space that assigns a real scalar to any pair of elements in that linear space with the following properties:
i. <f, g> = <g, f>
ii. <f+h, g> = <f, g> + <h, g>
iii. <cf, g> = c <f, g>
iv. <f, f> > 0 for all nonzero f in V