Saturday 16 May 2020

Matrix Decomposition to reduce Dimensionality of Matrices in terms of Principle Axes

Blindly any system can be defined by a Matrix as any system of equation formed with inputs and empirical outputs.

Determination of solution to these matrices is the Grant Goal of Linear Algebra.

Encountering Matrices naively as "solving simultaneous linear equation in school" is not possible as the no. of dimensional will be very large. Then how do we solve these.

Solution Approach:

We encounter this problem by "Divide And Conquer" Rule.

First and Foremost, we should ensure the vectors (columns from system of equations) are linearly independent otherwise all the below stuff is nothing but a fluff.

We first like to find the Eigen Vectors and Eigen Values of given Non Singular Matrix (i.e., consistent has one or many solutions).

Eigen Vectors are vectors which are inline with Transformation forces (system of equations), they only scale by a factor of corresponding eigen values.

We find the Orthonormal form of Eigen Vectors and determine the Principle Axes. These Orthonormal form are used forming basis of the Vector space to represent entire Matrix.

The Principle Component of the Matrix is Identified based on the Eigen Values. We give large important to largest Eigen Value.

But these procedures are not enough as not always we get Square Matrix. hence we have various other tools in our Arsenal.

Gram Schimt Orthogonalization - This Removes one basis component from other one by one and ensures that the eigen vectors are orthogonal.

We have different Matrix Decomposition Methods.
1. LU (Lower Triangular, Upper Triangular)
Divides the problem into 2 parts, makes the matrix independent of B.
2. QR
Q - Quadratic form of Matrix (Formed based on Eigen vectors and Matrix to represent the Principle Component)
R - Upper Triangular Matrix

3. SVD - UWV^T
For find the W (Sigma Matrix) which represent the eigen vectors and Principle axis together here.

SVD is the Epitome as it applies for both Square and Non Square Matrix. Dimensional Reduction is easily possible with it.

The Question then is why do we require other tools in our Arsenal?? For which problems they are Useful?

No comments:

Post a Comment

Meditation and 5 L

Relaxation and Laughter distinguishes being human and human being . Relaxation is meditation. May be it is a lie, but a beautiful one, whic...