The Geometry of Linear Equations
Written by: Ruwan Sri Wickramarathna — Senior Data Scientist at OCTAVE
The purpose of this section is to understand the fundamentals of Linear Algebra which is to solve a set of linear equations. More importantly, it is worth understanding how this solution lies in the N-dimensional space.
In mathematics, a linear equation is an equation that can be put into the form
Let’s say you have n linear equations with n unknowns. Solving an equation means finding the value of the unknown variables that satisfy all the equations. For example, if x + y = 0 and 2x-y= 6, then the solution would be x=2 and y=-2.
There are multiple ways that you can represent a set of linear equations.
- Matrix Form: This represents the equations in the algebra format.
- Raw Picture: This represents one equation at a time and then checks for the lines to meet.
- Column Picture: This represents column at a time and check for the vectors.
Ex: 2x-y = 0 and -x+2y = 3
Matrix Form
A: coefficient matrix, X: unknown vector and b: Output Vector. Final representation AX = b
Raw Picture
Here we have plotted the two linear equations and the solution lies where the lines intersect. i.e., we plot the 2x-y=0 line and -x+2y=3 in the Cartesian coordinate system. The solution for the equation would be the intersection point of two lines.
Column Picture
Here it is asking to find the right linear combination of the columns which satisfies this equation. The idea is that we have two column vectors, and we need to multiply them by some numbers (x, y) to get the right-hand side column vector.
Remember the columns are vectors, i.e. orange line represent the (2,-1) vector and blue line represent the (-1,2) vector.
The goal of the column picture is to find the right magnitude for the vectors which gives the right-hand side vector. Here the values are x=1 and y-2. i.e. one times orange line plus 2 times blue line equal to the right hand side vector (0,3).
Since we knew the answer, it was easy to represent this in a column picture. But what if we didn’t know the answer? If you try all the combinations of x and y, then there will be millions of vectors filled in the whole plane. Those are all possible linear combination that we can have, but there will be only one solution which satisfy our requirement.
When the number of unknowns increase, it is difficult to visualize the linear equation in raw picture.
ex: let’s suppose 2x-y=0, -x+2y-z=-1 and -3y+4z=4 are the linear equations.
Then this can be represented in matrix form
In 2-unknown variables linear equations each equation represents a line, but here it’s going to be a plane since we have 3 axes. Therefore, each equation represents a plane, and the solution would be the point where all 3 planes intersect. However, as you can see here it is difficult to identify the intersecting point. Therefore, the raw picture not going to be a good representation of the higher dimensional linear equations.
In the column picture now, we have 3 columns to deal with,
Then the initial vectors might be something like this,
You can understand that it is going to be a nightmare to find the correct (x, y, z) numbers by plotting the vectors.
The raw picture and column picture visually shows how the solution has been derived from the linear equations. But that is not a feasible approach to solving any linear equations.
Solution for the linear equations
AX = b is our general form to represent a linear equation. It is almost clear to you that once the right-hand side(b) changes the solution will change. However is it possible to find a solution for every b. In simple terms can all the linear combinations of the columns fill the entire N-dimensional space? That is the big question. The answer is Yes and No. For some linear equations, it is possible, but for some, it is not possible. i.e., If the matrix is singular or non-invertible then you cannot find the solution for every b. ex: let’s assume that two equations are similar in n-dimensional linear equations, then there won’t be any additional information added by one equation. That means it is going to be an (n-1) dimensional plane in n-dimensional space.
In the conclusion, even though raw picture and column picture gives a visual approach to solving the linear equations, it is not feasible when we have more than 2 unknowns. That is where the matrix form becomes much more feasible.
