• Published 1st Nov 2012
  • 1,910 Views, 16 Comments

Theoretical Physics is Magic - Smeapancol

In return for showing her the magic of friendship, Luna teaches Twilight to understand physics.

  • ...

Day 2 - Vectors

[Best viewed with 'small' size text and 1.5 size spacing.
If the equations seem out of order, reload the page and they might be fixed by then.
For a pdf version of this chapter with better formatting, please click here.]

Unit 1: Space

Lesson 1- Algebra

Day 2- Vectors

Luna had not said where their meetings were to take place, so Twilight hoped she was going to the right place by going to the Canterlot archives again. She did not want to anger Luna by not showing up, as she was never quite sure how the goddess, who had after all spent 1000 years an the moon without interacting with other ponies, would react to anything.

She heard a commotion as soon as she entered. Dreading what she might find, she sped to a gallop and raced towards it. The center of the library was ransacked and a tornado of books whirled above them. Luna was right below it with a number of cowering librarians in front of her.


“Princess Luna!” Twilight screamed, hoping she would be heard over the gale.

When Luna looked over and saw her the tornado stopped and all the books piled themselves into very tall stacks that reached near the ceiling.

“Hooray, my new disciple is nigh!” Luna reared with joy before collecting herself and resuming her ordinary stiff, regal demeanor. “These ponies were just explaining the Dewey decimal system to me.”

“Hahaha, yes! I saw, Princess. I think next time if you just ask for the book you want she’ll just get it for you.”

Luna nodded. “Oh! That would probably be more convenient for everyone.”

“I think so! Um, in fact, maybe next time we should meet in Canterlot Castle instead of over here.”


“Shh! This is a library!”

“Sorry. Anyway, let us get down to work!” said Luna as she conjured a blackboard once again. A block of chalk appeared before them and Luna broke off a small piece with her telekinesis. “The first thing to learn, if you want to do physics, is Linear Algebra.”

“Um, what about everything we did yesterday?” asked Twilight.

“That was the zeroth thing you need to learn. Linear algebra give us the right way to add and multiply the kinds of mathematical objects that represent physical quantities.”

“Aren’t physical quantities just numbers that are multiplied in the ordinary way?”

“They can be, but not always. There are some things in physics that are just ordinary numbers, but more generally they are collections of numbers, which are called tensors. Only whole tensors can be multiplied and only according to certain rules.”

“Why is that?”

“It all follows from the concept of space. It has to do with representation theory and the local coordinate invariance of space. I will show you all about that eventually. Anyway, just trust me for now.”

“Very well, Princess.”

“And of course derivatives and integrals are linear operators as well, but they act on function spaces, which are a more abstract kind of vector space.”

Twilight felt like she was already dreading the lesson.

Definition 2.1 : Morphism

“Linear algebra is about vector spaces and about functions on vector spaces. For every kind of object in mathematics that has operations defined on it, like addition and subtraction it is also possible to define morphisms between different objects. These are functions mapping one object to another which preserve the object’s characteristic operations. Did that make sense?”


Definition 2.2 : Linear Map

“Well nevermind because you only need to understand linear maps for now. These are functions between vector spaces that preserve vector space operations. In other words, a function A:VW is a linear map between the vector spaces V and W if



It is clear, first of all, that linear maps only exist between vector spaces over the same field, since in this equation, a has to be interpreted as an element in the fields over which both V and W are defined.

Now the interesting thing about linear map is that applying can be treated as a kind of multiplication. Do you know how you would prove that?

Twilight went to the board and levitated a bit of chalk. “I’d have to show that function composition obeyed the three properties of multiplication. Left distributivity is given by assumption as the first property of linear maps. Right distributivity would require showing something like this.


but I do not even know what A+B would mean.”

“Is the sum of two linear maps, as you have defined it, a linear map?”

“If we take equation 2.3 as a definition, then it looks like it satisfies the right properties.”

“So,” said Luna with a satisfied smile, “you have learned that A+B is a linear operator. But is it really a kind of addition?”

“Associativity and commutativity clearly hold because the addition of operators in defined in terms of the addition of vectors, so necessarily these properties carry over.

There’s also an additive identity. That would be the map that sends every vector to 0.”

“Which can be written as 0.”

“Yes... as previously noted. And there are inverses too, because you can just define (-A)(v)=-A(v). So there is a kind of linear operator addition and right multiplication by vectors distributes over it!”

“You have learned something important just now. You started out by writing an equation about adding functions, which for all you knew was meaningless, and asked, ‘is this true?’ You then showed how to interpret the equation and asserted the reality of what you had discovered. That is one of the fun things about mathematics; anything that is meaningful is real. Isn’t that wonderful”

“Sometimes I don’t know about your philosophy, Princess.”

“Of course it is wonderful, Twilight Sparkle! In any case, we know about left and right distributivity. What about associativity?”

“I’d have to prove something like this.


But I don’t see how-oh well it’s obvious. That’s just the definition of function composition, so of course that’s true. It’s also a statement of associativity for multiplication of A, B, and v.”

“Exactly. It has nothing to do with linearity.

So now we do not have to write A(v) anymore. We can just write A v and say that v has been multiplied by a linear map called A, which obeys the properties s


They distribute over addition (which is true of all multiplication) and they commute with scalars.

“This is confusing. We’re looking at the same thing in two different ways at the same time. A function is something you do, but we’re looking at the function as more of a thing now.

“That is one of the tricks to getting really good at mathematics. Once you can think of something as two different things at the same time, you know you are getting somewhere!”

Twilight was approaching her limit. She held her head and slowly shook it back and forth as she tried to let the philosophy settle in. “Ooohhhhooohhh...”

“Do not let your brain shut down on me now, Twilight Sparkle! We are only getting started!”

“Please Princess... no more of your philosophy today at least.”

“For the rest of the day only proofs. Promise! The issue that will preoccupy us for now is to characterize the linear operators, especially what we can say about their invert ability.

Now, think of an expression that looks like this:


Definition 2.3 : Linear Combination
This could be finite, or it could go on forever. Each is a vector and each is a scalar that goes with the corresponding . I have taken a set S of vectors that includes all the , multiplied each one by its own scalar, and then summed them all together. This is called a linear combination of S.

More formally, you should think of a linear combination as being given by a map from a set of vectors to a set of scalars. Expression 2.6 is more of a mnemonic, but there are linear combinations that could not be written in that way.”

“Why not?”

“Because I have written the expression as if S was a countable set, but it might be uncountable.”

“What does that mean?”

Definition 2.4 : Countable and uncountable

“Oh dear, oh dear... well you asked for it! Countable means that a set can be given as a finite or infinite list, like the list describing the sum in expression [number]. Essentially a set is countable if it can be objectively mapped to the natural numbers. A set is uncountable if it cannot.”

“Are you saying that there are degrees of infinity?” asked Twilight skeptically.

Proposition 2.1

“Precisely so. There are sets that are bigger than the natural numbers. Just as an example, I will show you that the range [0,1) of real numbers is uncountable.

The proof is quite simple. It is a proof by contradiction. Let us say we have an infinite list þ of real numbers which, we hypothesize, contains all of them. We can write it out in decimal form like so.

It is always possible to construct a new real number n which is not in the list. Start with the first digit after the decimal point of n. Make sure it is different from the first digit of the . Then you know, whatever the rest of the digits are, that it cannot be equal to the . Next, make sure the second digit of n is different from the second digit of . You then know that the new number cannot be equal to or . Do you see what happens?

“If you continue this procedure all the way down the list, you always end up with a real number that’s not equal to any element of þ!”

“Right! This disproves the hypothesis that the þ contains all the real numbers. And since we made no restrictions upon þ, this proves that no list can contain all the real numbers.”

Twilight groaned. “I thought you promised no more philosophy!”

Luna looked perplexed. “This is not philosophy. This is just the ordinary real world!”

“Those are numbers, not reality.”

“Close enough! Just as an aside, I will tell you that the integers are countable and so are the rational numbers. You can think about how to prove that yourself if you want, since you do not actually need to know that.”

“Alright.” Twilight hoped she would never have to think about it again.

Definition 2.5 : Spanning set

“Back to linear algebra. The thing to talk about is the span of a set of vectors. Think of an ordered set of vectors , called the spanning list, and think of all linear combinations of those vectors. This produces a vector space called the span of S, which I will call V.”

“Why is it important that S be ordered?”

“Because now we can specify every element of V as an ordered list of scalars. The ordering defines the mapping. That is why, in three-dimensional space, you may write vectors as

where the are all real numbers.”

“This is a very convenient way to think about a vector space. Normally something like this will define almost every vector space-as the span of a set of vectors. Similarly, it will be much easier to think concretely about linear maps. We just have to think about how the linear map acts upon a spanning list.

By the way, I have not proved that V is a vector space, but can you see why that is?

“It’s obvious.”

Definition 2.6 : Linear independence, basis

“Good. Next we need to talk about linear independence. Now suppose I had a linear combination of S with the condition that

Clearly there is always a trivial solution to this equation if all for all i. What we are interested in is a nontrivial solution. In other words, for some i, . If there is such a solution then S is said linearly dependent. Otherwise, it is linearly independent. If a spanning set S is linearly independent and spans a vector space V, then S is called a basis for V.”

Twilight nodded.

Proposition 2.2

“The nice thing about a basis is that every vector in the space it spans is uniquely given as a linear combination of the basis.

Now assume there are two distinct linear combinations, which I will write as and , which produce the same vector w.

I can just subtract one linear combination from the other to get zero.

which proves that S is not linearly independent. You see?”


Corollary 2.3

“This also proves that if S is a list of vectors and an extra vector w in span(S) is appended to S, then the result will no longer be linearly independent. There will be two ways to produce w: itself, and, because w is in span(S), there must also be some linear combination of S that produces it.”

Twilight nodded. “But I need to develop some intuition about these. Can discuss a few simple examples?”

“Very well. Hum. Well think about the vector space ℝ. Are the vectors 1 and 2 linearly independent?”

“Of course not, since 0=2*1-2.”

“And clearly, it would be impossible to have a spanning set with more than one element in ℝ.”

“Yes. For any two numbers there’s a linear combination that results in zero.”

“What about ? Try the vectors {1,0}, {0,1}, {1,1} and {2,2}.”

“Those vectors are not linearly independent because the last two point in the same direction.”

“And if you removed the last one?”

“Then you could still form the third vector by summing the other two.”

“And if you removed the third vector from the list?”

“Then the vectors would be linearly independent because they all point in different directions.”

“I think you are getting the hang of this, Twilight Sparkle. What about ?”

“It seems like you could have three linearly independent vectors. So {1,0,0}, {0,1,0}, and {0,0,1} are linearly independent, but if you added any other vector to the list, they no longer would be. Hm. I just noticed there may be some other vector spaces hiding in here. Let , , and be these vectors. Suppose I let . Then and on their own form a linearly dependent set. This means that and are both linearly independent, so neither of them can have a nonzero factor in any linear combination that is equal to zero.”

“You are really getting the hang of this! Yes, you can always divide a list of vectors into a vector space which is linearly independent and another which is linearly dependent.”

Twilight smiled. She really did feel like she was understanding! “It seems like you can’t have more linearly independent vectors than the dimension of the space because that’s the number of vectors you need to span that space.”

“Quite so! But actually we haven’t defined the concept of a dimension yet, so you’re jumping ahead.”

Twilight frowned. “But aren’t you already admitting implicitly that this is so by writing all the vectors in ℝ as single numbers, and the vectors in as lists of two numbers, and allowing me to write the vectors in as lists of three numbers? Clearly the reason that notation works is that the list represents a linear combination of a spanning set of vectors, and if you weren’t sure you didn’t need any more than the dimension of the space, you would write a longer list!”

Luna narrowed her eyes. “I admit nothing.”

“Oh come on!” Twilight rearedfore hoof.

Luna waved her forehoof. “You have passed my test!”

“What test?” asked Twilight with annoyance.

“I merely reprimanded you to test your intuition. You have enough to proceed.”

“Right... a test.”

Theorem 2.4


“Shhh!” Twilight covered her face with embarrassment.

“Well that is the most important theorem!”

Lemma 2.5

The first thing to prove is that for every linearly dependent spanning set S, there exists a vector which can be removed from the set without altering the vector space it spans. Let us write

to denote a linear combination of S. That good?”

Twilight nodded.

“You are not going to go crazy on me, are you, Twilight Sparkle?”

Twilight was struck by the irony that Luna would be worried she might go crazy. “What exactly are you iterating over?”

“I just wrote i under the sum symbol to say that we sum over i. We do not worry over how big S is, so you can assume that the sum is over whatever values it needs to be.

“Now let be nonzero value and write

This proves the lemma. As to this notation, there is no good way of saying you want to iterate over some set except skip one element of it. The summoned says we sum over i such that ij. How annoying. Next what do you think you could do?”

Twilight stared at the equations for a second. “We want to show that the span of is the same as span(S), so to do that we’d have to show that we’d have to show that every vector which can be a linear product of S is also a linear product of .”

“Good! Now try to do that.”

“Say that there was a vector w which is a linear combination of S. Let’s write

Now I can substitute in the expression for .

and this shows that w can be written without relying on at all. QED, Princess!”

Proposition 2.6

“And look what a complicated-looking equation you’ve written!”

“I did, didn’t I?” Twilight said gleefully.

“Now here comes the proof that everything hinges on. In a vector space V, every finite linearly independent list is smaller than or equal in size to any spanning list of V.

Let S be a linearly-independent list of vectors (though not necessarily one that spans V, and let U be a list of vectors that spans V.

Now, if we remove an element from S and insert it into , we can be sure that is linearly dependent by corollary 2.3. Therefore there is some element that can be removed from without changing the span.

Next, remove another element from S and insert it to produce . Once again, must be linearly dependent. Since all of the in are linearly independent, so we cannot safely remove any of them, but there must be some that can be removed from without changing its span.

Continue the process, adding an extra element from S to with each step and removing an element of U. What happens? First suppose that S is finite and S>U.”

“In that case, the process must terminate at some step n such that has no more elements of U left in it.”

“But we know that is linearly dependent because 2.3, right?”

“So there’s a contradiction. Therefore, either S and U are both infinite...”

“In which case in which the process never terminates and nothing is proven.”

“... or S is finite and SU. Which proves the proposition.”

“Right! By the way, what can we prove if S and U are both finite and of the same size?”

“Then the process terminates at some .”

“Which proves that S is a spanning set! That proves part of theorem 2.3. To prove the rest, suppose U is a finite basis for V. Can S also be a basis?”

“Only if S is the same size as U.”

“If S is a finite basis, what does that say about U?”

U is also a finite basis of the same size as S.”

“So every finite basis of V must have the same size and every linearly-independent list which is the same size as a basis must also be a basis.”

Definition 2.7 : Dimension

“I hope I can finally talk about the dimension of a vector space now!”

“You can talk about the dimension of a finite-dimensional vector space now.” Luna said sternly. “For shorthand we can write dim(V) for the dimension of V. Now, what if U and S are both bases for V?”

Proposition 2.7

“Then the issue is more subtle.”

“Exactly. So for an infinite-dimensional vector space, it is possible that there could be two bases, one of which is countable and one uncountable. We will see some examples of that eventually.

“So infinite-dimensional vector spaces don’t really have a very clear dimension!”

“No, the most you can say is that their dimension is infinite, but it does not necessarily settle on any kind of infinity.”

“I think I need more intuition about that.”

“Not now. We won’t need any infinite-dimensional vector spaces for a while!

The important thing is that we now we know how to think of a vector as a list. You can define some basis of the vector space, and write vectors as a list that describes a linear combination on that basis.”

“Of course, I already did know that...” Twilight said under her breath.

Definition 2.8 : Image, kernel

“Now let us think about linear maps. Let A:VW be a linear map, and let V be a finite-dimensional vector space with basis B. I will define A B to refer to the list of vectors that results when each is multiplied by A. We refer to span(A B) as the image of A. The image is a subspace of W. And the kernel is the set of vectors in V which A sends to zero.”

“Got it.”

“We can write ker(A) for the kernel of A and Img(A) for the image of A. It should be easy to see that the kernel is a vector space.”

“That would follow easily from the definition of left distributivity for a linear map.”

“Now if A B is a basis for the image of A, then necessarily A B is linearly independent, whereas if some nonzero vector in V was sent to 0, then A B could not be linearly independent because there would be some nonzero linear combination of it that is sent to zero, right?”


Proposition 2.8

“So A is only invert able if its kernel is zero-dimensional.”


“Now let us define the vector lists P and Q such that A P is a basis for the image of A and Q is a basis for the kernel of A. Clearly P is linearly independent because otherwise A P could not be, and clearly PQ must be linearly independent because, if not, then there would be some nonzero vector v such that v is in the kernel of A and A v is in the image of A, which by definition is impossible. Also, PQ must be a basis of V because otherwise that would imply a nonzero vector w which is not in the kernel of A and such that A w is not in the image of A.

Proposition 2.9

This proves that the dimension of the domain of A must be equal to the dimension of the kernel of A plus the dimension of the image of A.”


“Linear algebra is becoming simpler by the minute. It’s all very constrained.”

Definition 2.9 : Invertability

“Indeed. The last topic for today is invertability. A linear map A is invertable if there is a linear map Z such that Z A=1 and A Z=1. We do not like to worry about cases in where there is only a left inverse or only a right inverse, so we say that both have to be inverses of each other. By the normal rules of multiplication, we know the inverse is unique.”

Proposition 2.10

“And we also know that V and W must both have the same dimension, since both A and Z must have a zero-dimensional kernel!” said Twilight with glee.

“Right! This implies that both A and B are injective and surjective. Injective is the same as saying they both have a trivial kernel and if either were not surjective, then it could not be that the inverse was injective.

Definition 2.10 : Isomorphic

Now we say that two vector spaces are isomorphic if there is an invert able linear map between them. Two vector spaces which are isomorphic can be treated as the same, because you can think of either one as the other one, just with a map applied to it. Let us show that two finite-dimensional vector spaces are isomorphic if they have the same dimension.

Let A:VW and Z:WV. We know that every finite-dimensional vector space has a basis already, right?”

“Yes. That was in 2.6.”

“Oh yes. Now then, let be a basis for V and let be a basis for W. Now we can define

Clearly A and Z as defined are both linear, and since we know that every vector has a unique representation as a linear combination of a basis, this definition is not logically contradictory either. A and Z as defined are both inverses of one another, and therefore V and W are isomorphic!”

“This means that there is only one vector space of any dimension for every field!”

“Yes, so like , , and are pretty much what we are talking about, at least for finite vector spaces.”

“Wait a minute... what’s ?”

“Oh that... nothing! You’ll hear all about it later!”

Luna winked and disappeared. Twilight blinked and hoped she would be able to keep herself out of trouble.