Moments

by Bad Horse

The von Neighmann-Maregenstern theorem

It’s a beautiful day, if you like that kind of thing.

Again I congratulate Holly Copter. Again I flatter Chatter Box. Again Derpy bobs her head up and down in excitement at my news. The lies flow smoothly off my tongue now. My smile lays soft and guiltless over my face. I disgust myself.

Again I buy the watch from Old Times. Again he stands in the corner and smiles absently. Again I give the watch to the colt, and he brushes his whiskers over the silver case’s scrollwork in awe.

Across the street, a middle-aged stallion runs a hoof absent-mindedly over the ragged hem of his shabby tweed jacket as he pauses to watch his princess pass by. For twenty years he’s fact-checked the mayor’s monthly reports before they’re sent on to Canterlot. For the past five years these reports have come directly to me. They’re accurate and reliable. Until today, nopony has ever thanked him.

I hurry away down the street without making eye contact. I never speak to him anymore. He’s tired, so his gratitude on being given the watch is slightly less than the colt’s. He wastes precious seconds stuttering when he speaks, which he always insists on doing.

I’d rather give the watch to the colt most of the time, and to the stallion across the street the rest of the time. But the von Neighmann-Maregenstern theorem proves that, given four reasonable assumptions, the right policy is to always give the watch to the colt.

The mare with the glasses and bobbed mane hurries up to me as I hand out pastries. She does this every time, shamelessly, not caring that some ponies here have never gotten one. In the grip of my magic, the pastries are the size and shape of grenades. I toss her one and smile.

Again I send Rainbow on her fool’s errand. She’d kick my purple butt good if she found out. Again I let my friends trot to their last moments in happy ignorance instead of treating them with the dignity I’d expect myself.

Again I avoid Applejack. Again I try to persuade Apple Bloom, my charge and my unsuspecting confessor, of the rightness of my actions.

The fourth assumption of the von Neighmann-Maregenstern theorem is that if M is better than L, then a fifty percent chance of M and a fifty percent chance of N is better than a fifty percent chance of L and a fifty percent chance of N, no matter what N is. Seems obvious, doesn’t it?

Say M is giving a pastry to Pinkie Pie, L is giving it to Rainbow Dash, and N is kissing Big Mac on the lips. Toss a gold bit in the air. If giving a pastry to Pinkie is better than giving one to Rainbow, then giving a pastry to Pinkie if it comes up Celestia or kissing Big Mac if it comes up Luna is better than giving a pastry to Rainbow if it comes up Celestia or kissing Big Mac if it comes up Luna, regardless of how you feel about Big Mac.

But what if N is also giving a pastry to Pinkie?

The fourth assumption then says that giving the pastry to Pinkie Pie all the time is preferable to giving it sometimes to Pinkie and sometimes to Rainbow. But that’s the question I used the theorem to answer!

The theorem doesn’t prove that always giving the watch to the colt is the best thing. It assumes it.

“And what’re you gonna do about it?” Apple Bloom asks.

Nothing, ever. By the time I realized, it was too late.

I don’t think Apple Bloom knows about the circularity in the von Neighmann-Maregenstern theorem. But her eyes say that I’m hiding something. I wonder what it is.

“I need a hug,” I tell Big Mac. He shelters me, and shushes my crying, while I prepare to force myself on him, again, knowing I’ll never have to face the consequences.

I should just let him die. Let him finally die, not play out my games and my fantasy for eternity. He deserves to be allowed to die.

We kiss. He pulls away and stares, not angry, just confused, as if I were a pig trying to fly. I look away, and wish the asteroid would hurry.

I deserve to die.

So why don’t I? Why don’t I just collapse here in the road and cry, until the sky glows white and the treetops and my fur ignites and then nothing—

Oh, I remember. Because I can’t.

Each time I cast the spell it hurls my memories back in time, forcing them onto the earlier Twilight Sparkle. By the time I realized the flaw in the von Neighmann-Maregenstern theorem, it was too late. I had changed nothing, learned nothing, in so many cycles that my brain’s neural connections had already provably converged to a steady state. I am physically and mentally identical now each time I begin again, and so is the rest of the world. I’ll continue to give the colt the watch, and lie to my friends, and use Big Mac, for all eternity. That’s science. I’m as stuck as a fly in amber. I can’t change a thing.

But still… what if I could?

Something in the back of my head rises up and casts a black fog of panic over the thought. As I look up at the sky and cast the spell, I suddenly realize what I’m hiding, and—

Comments

[Jake The Army Guy]

SO... wait. If she forgets everything every time she loops, then how is she narrating?

[terrycloth]

3877264 No no no, she's saying that she stopped learning new things because the addition of each identical loop to the endless array of identical loops before it has stopped changing her mental state. She doesn't forget, but the last loop that's getting added is nothing new.

[FashionablyHospitable]

WAT

NO

I THOUGHT THIS WAS GOING TO BE A BALM, NOT A KICK

[Jake The Army Guy]

3877289

So... why does she say she can't just end it? I thought it meant something about how the loops affected her won't let her change the decision?

[terrycloth]

3877464 If she was going to change her mind she would have changed her mind while her memories were still changing. At this point all she remembers are identical loops. Nothing about her situation is going to change, ever, so why would her decision change?

It's not literally true. Maybe realizing that she's trapped in an endless cycle and has given up hope is the change that'll make her change her mind?

[Pyro The Reader]

3877601 This fic is starting to get too complicated...

[Jake The Army Guy]

3877601

That... but, but that...

Eh, what 3877752 said.

[Dusty Sage]

Blast you, BH, you've sent me running back to Theory of Games and Economic Behavior.

[Jordanis]

3877398

Ho ho ho, you're reading a Bad Horse story, my lad!

He's bad, you see.

[Blue_Paladin42]

3877398
Boy?

You thought wrong.

[Alondro]

3875761 >>>The most scientifically consistent hypothesis is that the whole raising the sun / nightmare moon thing is an elaborate prank or game by the sisters.>>>

You just get through saying science doesn't apply... and then try to infer scientific likelihood.

Uhm... THE PONIES CAN TELEPORT. They HAVE TRAVELLED THROUGH TIME. They CAN USE TELEKINESIS. They have GIANT MONSTERS MADE OF STAR STUFF. Pegasi fly with teeny wings. They break the sound barrier. And there are dragons that breathe fire.

None of these things is remotely scientifically plausible, and this isn't even a fraction of the impossibilities we've seen in Equestria.

Oh, also Equestria IS noted to be spherical in several places and we've seen a globe in an episode or two.

So, as I was saying, they have magic. Lots of magic.

Push the damned rock out of the way, or at least invent a sensible in-world reason they can't do anything. Like maybe the asteroid is a block of anti-magic and their magic all dried up. Buuuut, then there's the problem that Twilight wouldn't be able to use her time spell or illusion or whatever she's doing in the story, because that's magic...

... but all magic is just as impossible as the alicorns moving the Sun and Moon... so your story invalidates your own argument.

Seriously, trying to say moving the Sun and Moon is scientifically impossible when your own story has Twilight doing something as equally scientifically impossible is ridiculous.

[Southpaw]

Beautiful work, Mr Horse. I loved Twilight in this, responding to events with calm - instead of her usual mindless panic - and a desire to spread an infinite amount of happiness in the time she has.

Happy or bleak, I'm very interested to see what you have in mind for the rest.

3881336 Okay, so how about this?

Luna went mad / always was mad and set the asteroid/comet/small star on it's course to finally bring her "eternal darkness" to Equestria. And at the critical moment, she sabotaged an unknowing Celestia's efforts, destroying them both minutes before the world-consuming collision. It's dark, and it's an explanation, but we really don't need one.

Given so many inconsistencies in the show that still drive canon, we have plenty of room for explanations of just about anything. The fact that BH didn't give us one certainly doesn't close the door for one, it leaves that door wide open. If he decides not to share that explanation with us, then that's his prerogative as the author. Just sayin'...

[TheJediMasterEd]

Be-
fore
you
slip into
Unconsciousness
I'd
like to have a-
nother kiss
a-
nother
flashing
chance
at bliss
Another kiss...
Another kiss...

[Bad Horse]

3881973 Very appropriate. You know that's a Doors song?

[Bad Horse]

3881336 Ponies can do everything; therefore, nothing can ever threaten them. That's why every adventure episode ever has problems that the ponies could have solved easily with magic. Why don't you take the show's writers to task for their logical oversights?

[TheJediMasterEd]

3882054

You know that's a Doors song?

WHAT THE DOORS WHO WERE THEY GRANDPA? ;-)

Yeah, I was eleven years old when Jim Morrison died of inhalation of vomit ("tragically not his own")

But seriously: I prefer the Hotrats' cover. (Even) more sturm und drang, you know?

[Bad Horse]

3882280 3874560 3874326
I'm being unfairly maligned! No one dies alone in this story. They all die at the same time.

[rogerwarlock]

3877264
I think after to many loops TS is just repeating the same. The only hope to break the cycle it´s the theory of chaos, just need 1 pony to do something a little different to start a new change.

[Bad Horse]

3878918 I'm glad someone recognized it.
All academic debate from now on must be stated in the form of pony stories.

[Bradel]

Wow, but this is up my alley. Probabilistic arguments about morality?

I don't think I have a whole lot that's useful to say in comments here, other than: this thing is going into my favorites right now.

[book_burner]

3882752

All academic debate from now on must be stated in the form of pony stories.

My time has finally come.

[mylittleeconomy]

I am always happy to read a story about ponies taking economics too seriously. In this case I don't think Twilight has it exactly right. If N is reducible to M, then you are making the claim that M is not necessarily preferred to L + M, which should be fairly obvious. Perhaps Twilight has a grudge against Rainbow Dash.

I believe the N-M utility theorem is supposed to be a description of rational behavior, not a guide to it. It will take your preferences as given, so all it can tell you to do is to consistently choose in accordance with your preferences. People do have some trouble being rational, however, so it could be of some use.

[Bad Horse]

5159433 (Re-replying to move my reply to the page for chapter 2, and to fix a couple of errors and rewrite a bit.)

After staring at this a long time, I think Twilight's reasoning holds up, but the logical chain is so long that I will probably never convince anyone of that.

If N is reducible to M, then you are making the claim that M is not necessarily preferred to L + M

Perhaps you're thinking that because N represents the same outcome as M, N+M = M. That's not how the notation works--each side of the inequality represents a probability distribution over possible outcomes, not a single possible outcome. The 4th axiom implies .5N + .5M > .5L + .5M, where each .5 is a probability, the probabilities on each side must sum to 1, and the possible outcomes are L, M, and N. The things being compared are the expected utilities of the two different probability distributions over outcomes given two different possible actions.

I believe the N-M utility theorem is supposed to be a description of rational behavior, not a guide to it. It will take your preferences as given, so all it can tell you to do is to consistently choose in accordance with your preferences.

People use the theorem to say that it's rational to simply optimize expected utility, rather than putting extra weight on low utility outcomes (risk avoidance) or on high utility outcomes (lottery playing). (People often claim the theorem explains risk aversion. What they mean is that you can explain cases where people optimize expected utility, but not expected dollars. But such cases are not actually risk aversion. All they are doing is denying that people really show risk aversion. That's what the theorem does--it denies that true risk aversion, meaning a preference for low-entropy probability distributions of utility, is rational.)

Some people then go on to say that expected utility maximization is rational, therefore public policy should maximize average personal utility. That's an incorrect use of the theorem, because the question of fairness should be incorporated into the utility function used to aggregate the utility of different individuals.

What Twilight did in chapter 1 is not quite that, however. She was making in each "run" a decision that gave her optimal utility as a one-time choice, but low utility in a repeated-choice situation because it was unfair.

(You could object that the theorem is meant to compare individual decisions, not policies: "If you want to compare adopting one policy across multiple runs, you should count as outcomes the total outcomes from all those runs, and your preference for fairness should be incorporated into the utility function you use to aggregate utility from those separate runs." But I don't think that's a good response; if the axiom of independence applies across possible outcomes of a single decision, surely it should also apply across the outcomes of different decisions.)

Twilight uses the theorem to talk about fairness instead of risk. "Accepting risk" means "accepting distributions of possible future versions of yourself where some of those future yous have very low utility." Twilight maps that onto "accepting distributions of possible people where some of them have very low utility." That is, accepting actions that lead to distributions where sometimes you are rich and sometimes you are poor, is like accepting actions that lead to distributions where some people are rich and some people are poor. Accepting the theorem's validity implies it is okay to do that.

The 4th axiom, the one used in that M+N > L+N inequality, says that rational agents can't value fairness. Suppose that P is the situation where Pinkie has the donut and Lyra does not, L is where Lyra has the donut and Pinkie does not, and M is where they each have half a donut. P > M > L, because Pinkie likes donuts more than Lyra does. But if the agent has a strong sense of "fairness", .5L + .5P > .5M + .5P, because the second distribution gives Pinkie 3/4 of the donuts over repeated runs.

The mapping from risk-aversion into fairness-aversion makes it easier to see that the 4th axiom directly causes the theorem's result. But mapping that reasoning back onto the question of whether rational agents can be risk-averse is so complicated that I had to stare at it for a long time just now to remember how it worked. That probably means nobody else understood it.

Consider the "safe" outcome distribution <1 x 2>, where you get utility 2 with probability 1, versus the "risky" distribution .5 x 5, .5 x 0, where you get utility 5 with probability 1/2 and expected utility = 2.5. VNM is interpreted as being a theorem which proves that a rational agent will choose the risky distribution.

However, I think you can derive this result from axiom 4 alone, which means it isn't a theorem; it's an assumption hidden within one axiom, which is rationalized by adding axioms and fancy math and making it appear that the result fell out of that, rather than having been built in from the beginning.

This isn't a formal proof, but an example of the kind of construction I think a proof would use: Break each distribution up into 6 equi-probable outcomes: <2,2,2,2,2,2> vs. <6,5,4,0,0,0>. (Justifying that you can always do this might possibly require use of the other axioms, in which I'm wrong about the risk-aversion conclusion, and something fishy is going on in my mapping it into fairness-aversion.)

Merge 1/6x2, 1/6x2, ... into <(2,2,2), (2,2,2)> = <.5 x 2, .5 x 2>. Merge (6,5,4,0,0,0) into <(6,0,0), (5,4,0)> = <.5 x 2, .5 x 3>. Now axiom 4 is sufficient to say we must choose the risky distribution.

Twilight reasons both about possible worlds, and about which person gets the utility, to virtually split the problem into 4 cases so that axiom 4 can be applied in the same manner. Let P = Pinkie gets the donut, R = Rainbow gets the donut, u(P) = Pinkie's utility for the donut, and u(R) = Rainbow's utility for the donut. Say Twilight uses an agent-independent utility function by imagining that after giving away the donut, she will become Pinkie or Rainbow with equal probability. Her utility calculations must sum up over all possible combinations of who gets the donut, and over whom Twilight becomes. The first policy says p(P) = 1. This would split into 2 cases, p(P, Twi=>Pinkie) = .5 and p(P, Twi=>Rainbow) = .5. But we break each of those cases in two by splitting up p(P) = 1 into p(P)=.5, p(P) = .5 (breaking up the same outcome into two cases). The second policy, p(P) = .5, p(R) = .5, naturally has 4 cases. We group the cases for both policies into 2 groups, each containing one case where Twi=>Pinkie and one where Twi=>Rainbow. We then apply axiom 4 and conclude, just from that axiom, that Twi must choose the unfair, "risky" policy p(P)=1.

[mishun]

But the von Neighmann-Maregenstern theorem proves that, given four reasonable assumptions, the right policy is to always give the watch to the colt.

Hmm, I think this is not always so here, and it has much to do with incomplete information and how exactly Twilight computes utility taking multiple loops into account (there are also standard utilitarian concerns and Twilight's personal satisfaction too, of course, but lets factor them out for now).

Suppose, Twilight goes through n loops, gets r_i of "reward" for each, and her final utility is u(r_1, \dots, r_n). It's reasonable to assume that u is symmetric and monotonous function. If she can deterministically predict consequences of her actions, then she just chooses one action that maximises reward and uses it for every single loop.

Now suppose Twilight has two accessible actions \{ a, b \}, but her knowledge is incomplete: either \{ r(a) = 101, r(b) = 0 \} with probability \frac{1}{2}, or \{ r(a) = 0, r(b) = 100 \} with probability \frac{1}{2}, and she has no way of improving her knowledge. Let her utility be:

u = \ln{\left( 1 + \sum_i r_i \right)}

Action a has better expected reward and expected utility of always choosing it is:

\frac{1}{2}\ln{(1 + 101 n)}

On the other hand, expected utility of choosing a half the time is:

\frac{1}{2}\ln{ \left( 1 + \frac{101 n}{2}  \right) } + \frac{1}{2}\ln{ \left(  1 + \frac{100 n}{2} \right) } \ge \ln{(1 + 50 n)}

Which is better for large enough n.
"Looping" part is important here because probabilities are not independent between iterations (in other case it would be optimal to always choose a).

[Bad Horse]

8899383
Thanks for the thoughtful comment!

The von Neumann-Morgenstern theorem is mathematically valid, so finding an exception would prove your exception didn't observe the 4 axioms of the theory.

In this case, you're applying a utility function above the level of the lottery. The von Neumann-Morgenstern theorem uses a framework in which utility must be additive across the different possible outcomes in a lottery. I applied the theorem assuming that each loop is a lottery, so you can't apply u across multiple loops.

If we were to treat the entire infinite series of time loops as one lottery, which seems to be your intent, then we aren't choosing between action a and action b, but between always choosing action a, and choosing a half the time. Then the entire infinite series of time loops is a single trial, and the single choice to choose a in half of those loops has higher expected utility because the u you chose imposes "risk aversion" (it gives higher utility to a more even distribution of outcomes). To apply the theorem then, we'd have to ask about a scenario in which we have multiple infinite series of time loops, and for some of those infinite series we always choose action a, and for other infinite series we choose action a half the time. We would then get the result that always choosing action a half the time (the choice with higher expected utility on a single trial, which is one infinite series of loops) would give the highest utility over the entire set of infinite series of time loops.

This nicely illustrates the problem with the theorem that I was trying to explain: The theorem is usually applied to argue that rational agents maximize expected utility, and this means that a valid utility function can't give preference to more-even distributions of utility. But it assumes that the utility function is an additive sum of p(X)u(X) for all possible outcomes X. In this case, it means assuming that Twilight's utility for an entire series of loops must be the sum of her payoffs in each loop, so you can't use u = ln(1 + sum(r_i). But that really is assuming the thing the theorem is being used to prove--that Twilight mustn't use a utility function that gives preference to even distributions of utility across multiple loops.

[Cosmic Cowboy]

The fourth assumption of the von Neighmann-Maregenstern theorem is that if M is better than L , then a fifty percent chance of M and a fifty percent chance of N is better than a fifty percent chance of L and a fifty percent chance of N , no matter what N is. Seems obvious, doesn’t it?

Is this explanation a little mixed up, or did I miss something between here and the illustration below it? Because they don't really match up, and this explanation here just doesn't make sense on its own.
If N could potentially be even better than M, then it'd be silly to assume a choice between M and L is always preferable to N and L, where the only difference is that M-L is a known relationship and N is variable.
The point made by the illustration is instead that since M is known to be greater than L, there's no benefit to be gained by giving them both spots in the choice. It's still not entirely sound unless N cannot have a known, fixed value, or else it would be disproven if N was known to be least of the three choices. But the theorem says nothing about N being an unknown, instead asserting that it doesn't matter what it is.

Am I missing something here? I say this not being familiar with real theorem underlying this chapter, only based off the fourth assumption as explained here, which is the only part of it mentioned here.

There's also a little confusion with what the actual question of the watch and how she originally expected the theorem to answer it for her, and consequently a bit of a jump to how the Double-Pinkie exception leads to this circulatory. Twilight's explanation here feels like it assumes at least some familiarity of the reader with the original theorem. Before I got to the comments, I was wondering why the fourth assumption was being explained first, and then confused when the first three were never even mentioned. This isn't an easy chapter for laymen on the subject.

[Bad Horse]

9985821
Thanks for reading and responding, but I can't parse what you've written.

If N could potentially be even better than M, then it'd be silly to assume a choice between M and L is always preferable to N and L, where ...

You wrote, "assume a choice between M and L is always preferable to N and L." That means, if anything, that getting to make a choice between M and L is always preferable to having both N and L That... is not talking about anything that I'm talking about.

... the only difference is that M-L is a known relationship and N is variable.

"M-L is a known relationship": Is that supposed to be M minus L? There is no "known relationship" in play; M, L, and N are all free variables. I haven't specified any relationships between variables except that M > L. (That's what "M is better than L" means in the utility-value-summing framework of the theorem.)

The point made by the illustration is instead that since M is known to be greater than L, there's no benefit to be gained by giving them both spots in the choice.

"there's no benefit to be gained by giving them both spots in the choice" -- Gained over what alternative? There IS no alternative; we're considering exactly one choice: that between getting a 50% chance of M and a 50% chance of L, versus getting a 50% chance of M and a 50% chance of N.

It's still not entirely sound unless N cannot have a known, fixed value, or else it would be disproven if N was known to be least of the three choices.

It's sound regardless of N's value. It does not matter at all whether N is the least of the 3.

1. "if M is better than L" means that M > L.
2. Let M = 6, L = 4, N = 0.
3. If you choose option A, you flip a coin, and if it comes up heads, you get $M = $6; if it comes up tails, you get $N = $0.
4. If you choose option B, you flip a coin, and if it comes up heads, you get $L = $4; if it comes up tails, you get $N = $0.
5. The only difference between these options is in the value of M and L. N has the same value in both cases.
6. Since 6 is more than 4, option A will always get you either exactly the same value as option B, or more value than option B, no matter how the coin flip goes.
7. This would still be the case if N were a million.