Throughout my school career, calculus and linear algebra were separate courses. So there were calculus problems, and there were linear algebra problems, and I figured they were separate branches of math. In my 2nd year of university, I did a course on vibrations and waves which required calculus and linear algebra as prerequisites. We solved a linear system using both, and I came out with a better understanding of both. It was like realizing complex numbers and exponents and trigonometry all belonged together as I did when someone told me about Euler’s magic formula. I want to solve a linear system with you. I will try to be as complete as possible, but in the end you will need knowledge of derivatives and matrices to get through this post.

Let’s take two masses and connect them to each other and to two walls with springs, all in a chain. So from left to right we have: left wall, left spring, left mass, middle spring, right mass, right spring, right wall. Let’s call the springs, from left to right, spring 1, spring 2, and spring 3. And let’s call the masses, from left to right, mass 1 and mass 2. Springs have things called spring constants, which approximates the relation between how much it is deformed and how much force it exerts. Spring constants are typically denoted with the letter k. Here’s a diagram of the situation.

So springs 1, 2, and 3 have spring constants , , and respectively. And masses 1 and 2 have masses and . This is a bit confusing because I’m using mass as a noun and a property. Usually no one has to say confusing sentences like that because the audience is assumed to know a mass has a mass which is denoted with an m. Also assume the springs are not compressed or stretched when the masses are at rest.

Another thing audiences are typically expected to assume is that the system is idealized. That means there’s no air, there’s no gravity, springs work exactly as the spring constant model suggests, there’s no relativistic mass, there’s no quantum considerations, and so on. We are only dealing with what is explicitly stated, and for things that are implied (such as the existence of time and space) we are assuming the simplest model we have for it, which is usually Newtonian. So if I move mass 2 to the right a bit and let go, the whole system will only jiggle in the horizontal direction. We don’t want to mess around in multiple dimensions because it gets messy and isn’t the point I’m trying to make. If you tried to do this with a real system, you’d need to give it some kind of rails, but my system is ideal so I don’t have to.

Let’s doctor the system a bit more for the sake of simple math. Let’s say and and and .

One more thing before we can start writing down the rules that govern the system. We need to denote the location and velocity of the two masses. Let’s call the displacement of mass 1 from its rest position , and say that positive displacement is to the right. We do similar with mass 2 and . So if I pull mass 1 a centimeter to the right of its rest position, will be 0.01 m. If I pull mass 2 a millimeter to the left of its rest position, will be -0.001 m. For velocity, we create and and they are the derivatives with respect to time of and respectively. For those unfamiliar with derivatives, they are basically rates. The derivative of chocolate with respect to price is how much more chocolate you will buy as the price increases (which is probably negative, because you’ll probably buy less chocolate if it becomes more expensive). The derivative of distance with respect to time is how much more distance you will travel as more time goes by, which is speed.

Let’s consider the force applied to mass 1. Spring 1 will pull it to the left if it’s to the right of its rest position, and push it to the right if it’s to the left of its rest position. So the force spring 1 applies to mass 1 is . Spring 2 is more complicated because both and determine the length of spring 2, which determines the force it exerts. If they are equal, spring 2 is at rest and exerts no force. If then spring 2 is compressed and pushes the masses apart. If then spring 2 is stretched and pulls the masses together. So the force spring 2 applies to mass 1 is . For mass 2, we have pretty much the same thing because the system is symmetrical, so I wont bother going through it. Anyway, we add the forces for each mass and get these two equations:

where and denote the total force applied to mass 1 and 2 respectively. Because we said , we can simplify this further to

Let's try this in the context of linear systems. We seek an equation of the form , where is a column vector of "states", is the derivative of with respect to time, and represents the relation between and . Knowing and have identical dimensions, we know must be a square matrix with matching size. First off, how many states do we have? This will determine the dimension of the matrices involved. You might want to say 2 because we have 2 masses. But this isn't the case. In fact, each mass represents 2 states because a mass can have a velocity independent of its position. So we have 4 states in total.

For , we know the derivative of with respect to time is just , but the derivative of with respect to time is acceleration, which we'll call . One of Newton's laws states , which we can rearrange to . Because is 1 kg, . So

Now let's fill in with our knowledge of and and, again, knowing that the derivative of with respect to time is .

Now, this is one of them differential equations, because we have a variable related to its derivative. In this case, is related to its time-derivative, . We already "know" what is, see above. But we seek another form where we can input a certain time, say, 3 seconds after we release the system from some state, and get the state at that time. There's lots of methods to find solutions to differential equations, but in the end, if you try something and it works, it works. This is how the methods were discovered in the first place anyway. So let's pretend for a moment that where is a matrix with the same dimensions as , is time, is the base of the natural logarithm (2.718…), and is something that will make solving the equation possible and will be interpreted later. Anyway, let's apply this and watch the math unfold.

here is the identity matrix, which is 0s with 1s along the down-right diagonal, and it basically acts like 1 in the matrix world, so that's why I can add it in the 5th step just above. I've also used 0 to denote a matrix filled with 0s with dimensions that match . In the last equation, we have two options: either the determinant is 0 or is 0. For those who want more justification for this dichotomy, check out matrix invertibility. If is 0, we get a really boring answer: the masses stay at their rest positions forever. This is a valid answer, but it is not the only answer. So we pursue cases where is 0.

decompose along the first row

decompose the left determinant along the first column and decompose the right determinant along the first row

Now this is just a quadratic equation. Let's create a variable to make it clearer.

Now, for those of you not familiar with complex numbers, you might be interested in my explanation in here. And for those of you not familiar with Euler’s magic formula, I encourage you to look it up because it is awesome, well-documented, and we are going to use some of its consequences. That’s right, in order to use linear algebra and calculus at once, we have to know how to use exponents, trigonometry, and complex numbers at once.

We have discovered 4 values of that make . This means we can find values of which make . Let’s look at the case where . Then we have

At this point we could multiply to get a single square matrix and use Gauss-Jordan elimination to solve. But because the form is fairly simple, I’d rather just “divine” the solution. Again, all 0 is not an interesting solution. We might try making some elements of 0, but rows 1 and 3 force 0s to come in pairs, and row 4 forces to be 0 when and are 0 and vice versa, so if there are any 0s, all is 0. So the first element will not be 0. Consider:

We can multiply our equation by any arbitrary scalar, which means we can make the first element whatever we want. If it’s the “wrong” value, we would just multiply by some arbitrary scalar to get to the “right” value. In other words, there are no right or wrong values for the first element. So let’s set the first element to 1. So . From row 1, we know . From row 2, we get . And from row 3, we get . We can do similar for the 3 other values of . Let’s differentiate between the 4 different values of with subscripts a through d. We do similar for the corresponding values.

By the way, did anyone notice we are finding eigenvalues and eigenvectors? Soon we find out what they really are. Remember this equation?

Let’s multiply it by an arbitrary constant .

Now, we can substitute in any and we want, so why not use all of them? We’ll have a corresponding constant for each. And then, why not add all those equations together? Then we’d get

We factor

Well now this is just the same form as so we can have

Remember, we only ever guessed at the value for , and this is just yet another guess. Only it happens to cover all possible solutions for . It’s hidden in differential equation lore, but basically if you have 4 states in a system, and you have a solution for the states which has 4 degrees of freedom, you have found the general solution. In this case, we can choose the constants through to be anything we want, so there’s our 4 degrees of freedom.

Let’s get back to reality. Say we pull the left mass to the right by 1 meter, holding the right mass where it started. I haven’t actually ever said how big the system is. We can say it’s really big, so that a displacement of 1 meter doesn’t cause the masses to hit the walls or each other. That way we can use pretty numbers like 1. We let go and say that moment’s time is . How will the system behave? Intuitively, you might say “the two masses will sway back and forth” or “the two masses will oscillate closer and farther from each other” or “both of those things you just said”. You’re good! Let’s see what the math says.

First we want to figure out what the arbitrary constants should be. We know the state of the system at . Namely, , , , and . So

Again, you can massage this into a system of linear equations and do Gauss-Jordan elimination, or you can just “divine” the answer. Looking at rows 2 and 4, I note that making and satisfies them. Looking at row 3 I note that . Finally, I look at row 1 and decide that .

So how does the system evolve in time? We get

Let’s concentrate just on the position of the left mass, . We have

Remember how I said we’d use consequences of Euler’s magic formula? We’re going to use the identity

So we get

Let’s apply the same steps to . We get

And now it’s time to interpret. Whoever said “both” before was right! But there’s more information here than that. We also learn that the frequency with which the masses sway is times slower than the frequency with which they oscillate closer and farther from each other.

So complex pairs of eigenvalues are like frequencies, and complex pairs of eigenvectors are like harmonic modes. If you let go of your bias against complex numbers, eigenvalues are exactly frequencies and eigenvectors are exactly harmonic modes. If you start the system from a state in which everything’s value is real, it will stay that way, but mathematically speaking there’s no reason you have to constrain yourself to real numbers.

Anyway, for those of you that haven’t seen linear systems before, I hope this leaves you with the same sense of awe I felt when I first experienced a linear system. For those of you who have seen linear systems before, I hope you appreciate how important it is not to artificially separate topics in math. Without a physical system to go with them, I find eigenvalues and eigenvectors entirely meaningless. Maybe at higher math levels, they take on meanings of their own, but I don’t think it’s appropriate to introduce them that way. Finally, as this was a very long and mathy post, I’d like to applaud anyone who made it all the way through.

This field was one of the ones which left me most baffled when I was a wee undergraduate taking physics classes, possibly because a combination of AP credits and double-majoring meant I was getting some of this before I’d had quite enough linear algebra, and partly because the instructors didn’t make clear the links between matrices and eigenvectors that were needed.

Once I

didunderstand the links the wonderful ocean of problems that were made not just simpler, but simpler in the same ways, really awed me.