Importance of matrix rank

What is the importance of the rank of a matrix? I know that the rank of a matrix is the number of linearly independent rows or columns (whichever is smaller).

Why is it a problem if a matrix is rank deficient?

Also, why is the smaller value between row and column the rank?

An intuitive or descriptive answer (also in terms of geometry) would help a lot.

Answer

A rank of the matrix is probably the most important concept you learn in Matrix Algebra. There are two ways to look at the rank of a matrix. One from a theoretical setting and the other from a applied setting.

From a theoretical setting, if we say that a linear operator has a rank p, it means that the range of the linear operator is a p dimensional space.
From a matrix algebra point of view, column rank denotes the number of independent columns of a matrix while row rank denotes the number of independent rows of a matrix. An interesting, and I think a non-obvious (though the proof is not hard) fact is the row rank is same as column rank. When we say a matrix ARn×n has rank p, what it means is that if we take all vectors xRn×1, then Ax spans a p dimensional sub-space. Let us see this in a 2D setting. For instance, if

A=(1224)R2×2 and let x=(x1x2)R2×1, then (y1y2)=y=Ax=(x1+2x22x1+4x2).

The rank of matrix A is 1 and we find that y2=2y1 which is nothing but a line passing through the origin in the plane.

What has happened is the points (x1,x2) on the x1x2 plane have all been mapped on to a line y2=2y1. Looking closely, the points in the x1x2 plane along the line x1+2x2=c=const, have all been mapped onto a single point (c,2c) in the y1y2 plane. So the single point (c,2c) on the y1y2 plane represents a straight line x1+2x2=c in the x1x2 plane.

This is the reason why you cannot solve a linear system when it is rank deficient. The rank deficient matrix A maps x to y and this transformation is neither onto (points in the y1y2 plane not on the line y2=2y1 e.g. (2,3) are not mapped onto, which results in no solutions) nor one-to-one (every point (c,2c) on the line y2=2y1 corresponds to the line x1+2x2=c in the x1x2 plane, which results in infinite solutions).

An observation you can make here is that the product of the slopes of the line x1+2x2=c and y2=2y1 is 1. This is true in general for higher dimensions as well.

From an applied setting, rank of a matrix denotes the information content of the matrix. The lower the rank, the lower is the “information content”. For instance, when we say a rank 1 matrix, the matrix can be written as a product of a column vector times a row vector i.e. if u and v are column vectors, the matrix uvT is a rank one matrix. So all we need to represent the matrix is 2n1 elements. In general, if we know that a matrix ARm×n is of rank p, then we can write A as UVT where URm×p and is of rank p and VRn×p and is of rank p. So if we know that a matrix A is of rank p all we need is only 2npp2 of its entries. So if we know that a matrix is of low rank, then we can compress and store the matrix and can do efficient matrix operations using it. The above ideas can be extended for any linear operator and these in fact form the basis for various compression techniques. You might also want to look up Singular Value Decomposition which gives us a nice (though expensive) way to make low rank approximations of a matrix which allows for compression.

From solving a linear system point of view, when the square matrix is rank deficient, it means that we do not have complete information about the system, ergo we cannot solve the system.

Attribution
Source : Link , Question Author : 0x0 , Answer Author : Martin Sleziak

Leave a Comment