Chapter 1

Part I

"For our struggle is not against flesh and blood, but against the rulers, against the authorities, against the powers of this dark world and against the spiritual forces of evil in the heavenly realms."
--Ephesians 6:12

The laptop case held a pistol.

The Smith & Wesson M&P9 fit easily, snug in the hole where Ryan had removed a slice from the underside of the case's foam padding. The top of the padding looked the same as before; the laptop still fit well, despite the gun. If you had pressed your hand against the foam all along the inside of the opened case, you would have felt the hardness of the weapon beneath; but Ryan doubted that security would go this far.

Ryan Szilard had worked the problem for only two weeks.

In the past, he thought, I would have enjoyed this. My heart would pound a little faster. I'd enjoy the prospect of breaking through someone's weak attempt at security. I would feel more alive.

He felt pretty much nothing, at the moment.

He opened the door of his car, stepped out, and walked towards the building.

It was about 2:00 in the afternoon, the hottest part of the day in the Washington DC metro area. The humidity was oppressive, and shimmers of heat rose from the car-park's concrete. The sky was blue and cloudless; the sun felt swollen and enormous in the sky. Sweat pooled in the small of his back after only a few seconds outside of the air conditioning. He was wearing a heavy pinstripe suit--he only ever wore a suit as a disguise, and he hated how it felt on him. He hurried to the doors of the target building; the sun reflected off on the mirrored windows of its twenty stories, making the concrete he walked across doubly bright.

Ryan had always preferred the night to the day, but one of the requirements had been that the break-in occur during the day.

When the automatic glass doors shimmed closed behind him, he surveyed the obstacles visible before him. Ryan was methodical. He had planned this quickly; but he had still planned it carefully.

First obstacle: Get through the card-operated turnstyle, while watched by the guards.

The cards necessary to operate the turnstyle were issued by the Department of Homeland Security--well, by a contractor who worked for the Office of Internal Affairs within the Department of Homeland Security. Regulations for this building required that the authentication information for the cards reside in a database physically separated from the internet. Szilard was currently about 98% confident that these regulations had not been followed; this was because he had illicitly inserted authentication information for the card in his pocket into the relevant database one week ago. There was the possibility that he had hacked into a honeypot, rather than the real thing. But he was reasonably confident that he had not.

He slid the laminate over the glass of the turnstyle. There was a beep, and a green light appeared. So far, so good.

Second obstacle: Show the guards his photo id, walk through a metal detector, and put his briefcase through the X-ray machine.

The hard part of making the card, of course, had been hacking into the database; making a realistic-looking laminated photo id was tremendously easy. The makerspace that Ryan belonged to had a machine that could print, complete with a shiny hologram-like strip, any ID card you wished. No one at the makerspace seemed quite willing to admit why they owned the machine. Ryan handed his card over to the guard without making eye contact, humming along to the music playing through his earbuds.

He was listening to music, of course. Guards were watching for people acting nervous. Someone listening to music appeared oblivious, and therefore not a threat. He chewed gum for the same reason. He hummed, just to irritate the guard a trifle, as the lyrics continued in his ears.

She scanned the card, looked at his face briefly, and handed it back. He put the laptop case on the conveyor belt leading through the X-ray machine, walked through the metal detector, and started browsing the internet on his phone as his briefcase went through the X-ray. He flipped through an online magazine about architecture as the briefcase slid through.

His laptop case had metal inserts in it. The composition of the metal was the precise kind necessary for blocking the entirety of the frequency of electromagnetic radiation that this X-ray machine used; he had determined what kind of X-ray machine was used simply by wandering by the entrance of the building a week and a half earlier. Because of the metal, his entire suitcase would appear to be a featureless, rectangular slab on the X-ray's screen. The gun would be invisible, hidden behind the metal shield.

There were, of course, problems with this.

As his case slid out of the machine, a guard approached him.

"Would you mind opening your briefcase, sir?"

Huh. Security was usually more lax. Ryan removed his earbuds.

"Excuse me?" he said, tucking the gum in his mouth behind the line of his teeth. "I didn't catch that."

"Would you mind opening your briefcase?"

"Oh, sure," he said, and unlocked and opened the case. His laptop, resting in a sleeve inside the case, was visible, as were a few pens and pieces of paper. Ryan helpfully lifted the sleeved laptop to show there was nothing beneath it.

"Thank you," said the guard. "That's quite a large laptop case."

"No problem, and thanks," Ryan said, reinserting his earbuds. He started walking away from the entrance, down a marble-panelled hallway. With each step, his Oxford-encased feet made noises that echoed briefly before dying away. He could see his reflection in the mottled marble siding. There were only a few other suits in the hallways; the lunch rush would have just ended--and people in this building probably worked through lunch, anyhow. His destination was on the same floor, so there was no need to worry about an elevator.

Part of the reason I don't care about this, Ryan thought, is that this is too easy. He had no individual opponent, really.

No one actually cared about security in a facility like this, despite the impressive apparatus. There was a security systems designer, who cared about making a system that would be approved and never get him in trouble. There was someone who had written the regulations the designer followed; there were contractors who carried them out, who did not care as long as their got their money. But no single person was responsible for the security, ultimately. Breaking into a system designed by a single person was more interesting; it was like reading an argument conceived of by a single mind, or fencing with a single person. A system like this was like an argument written by committee; it hardly held itself together firmly enough to shatter when you hit it.

After someone analyzed his break-in, they would come to the conclusion that no single person could be blamed for the flaw--which was precisely why he was able to break in, in the first place.

Not that you would care about this, even if it were more difficult, a little voice in his head said.

Oh, shut up, he thought.

He approached the third obstacle: A fingerprint controlled doorway, with another guard standing in an alcove near it.

As he walked towards it, Ryan could feel his pulse against the earbuds. Still the same as always.

He put his free hand into a pocket, and fit the rubber fingerprint-spoofing cap over his index finger. It was thin enough that the scanner would still feel the warmth of his finger; it was thick enough to cover up his own fingerprint, and replace it with the fingerprint of someone authorized.

The guard glanced at him as he approached, and he didn't bother to make eye contact. The guard was only here to prevent anyone from tailing someone else through the door.

He raised his rubber-coated finger and put it on the pad. The difficult part had been finding out what kind of fingerprint scanner was installed. Well, that and obtaining the fingerprint. Different scanners demanded different spoofing techniques.

The door unlocked, and slid open. He stepped through and it shut quietly behind him.

He was now in what was supposedly one of the most secure locations in the DHS. A part of him thought it should look much, much more impressive than it did; the hallway he was in looked like hallways in office buildings around the globe. There were no windows, true. The hallways seemed hermetically sealed, save for a few doors in it. But other than that, it was all very disappointing. He knew valuable information sat in the servers he could access through the doors to the left--part of him wanted to enter that room with a flash drive and bring some of it back with him. It would be valuable. Probably. Maybe. Fencing it would have been difficult as hell--but on the other hand, he didn't really care about money, right now, at least any more than he cared about anything else.

Anyhow, that wasn't part of the task.

Third door on the right, he told himself. His footsteps had grown quiet; this place was carpeted rather than tiled. Outside the door he stopped, and took a breath. I probably should try for some kind of big entrance, he told himself. Some kind of a surprise. At least a good, snappy one-liner. Nothing came to mind.

That's because you know you don't give a fuck about any of this, the little voice said.

He sighed, and opened the door without knocking.

Inside, a large, red-faced man with a prematurely receding hairline was working on a computer. The room was a standard, blandly-furnished office. He looked up at Ryan. His eyes lit with recognition; then with surprise; and finally with irritation. The man spoke.

"God. Fucking. Damnit. And damn you too, Ryan."

"It's good to see you too, Braden," Ryan said, and despite himself smiled a little.

"The security test had another four weeks to go, Ryan," Braden said, standing up. "You could have waited a bit longer. I had expected it."

"I had counted on that," Ryan said, and shrugged. He extended his hand to shake Braden's, but found Braden hugging him instead.

"How long has it been? I heard you were assigned to this test, and knew you'd get in somehow. But I haven't seen you in... I don't even know. It's been at least six months since I last saw you. At that barbecue I had. It's been forever."

"Yeah," Ryan said.

"How did you get in?" Braden said.

Ryan shrugged. "It wasn't really hard. I got into your contractor's card authentication database easily; I pretended to be a computer technician to some senior management, used his password, and everything was easy from there. For the scanner I got a fingerprint from a glass bottle used by someone I drank with nine days ago, and made a rubber spoof from that. It's all in the report."

Ryan set his laptop case on the desk as he was speaking, clicked it open, and handed a twenty-page report to Braden. Braden put it into a pile on the desk. Ryan adjusted the fit of his laptop inside the case, running his fingers all along the inside of the case. Then he snapped the case shut again.

"Of course it wasn't hard for you," Braden said, throwing the report onto an already structurally unsound pile of other papers. "I'm sure the report will be fine. How's working for the government been, though?"

Ryan shrugged again. "It's a job."

"You're managed by... what's his face, over at the Rockeville building."

"Michael Suprenant."

"Hah. That blowhard."

"Careful. He is your peer."

"I didn't hire him and he didn't hire me. I don't have to like him. He's a blowhard."

Ryan smiled, or at least pretended to. Braden spoke again.

"Are you enjoying the work that you get there? A lot of challenges?"

Ryan contemplated lying. But making up fiction was too much effort.

"Really... no. Most of what I do is bullshit. Python scripts can handle most of what I'm expected to do. I play games a fair amount."

"Huh," Braden said, and laughed. "Careful. I could tell your boss. He is my peer."

"Even so," Ryan said.

Ryan had been acquainted with Braden since undergrad. He hadn't really gotten to know him, though, till during grad school, when they had both studied computer science in a Master's program--Ryan had not approved of Braden's decision to get a government job afterwards, but Braden hadn't been consumed by the desire to create new, wonderful and world-changing artificial intelligences, like Ryan had been. Instead, Braden had apparently been consumed by the older and more traditional desire to make biological intelligences--he had married, and had picked a secure, undemanding job so he could pay attention to his wife and children. He now had... Ryan could not remember how many children. Three? Surely not so soon.

And of course I cannot complain about his unambitiousness now, Ryan thought. He himself had now had a year of government employment in information security. A year of staring at the ceiling and blowing spit bubbles and doing nothing. The ... event two years ago had broken him, and he was coming to doubt that time would ever remedy the injury.

But he didn't want to think about that now. He focused on what Braden was saying--he had been saying something.

"And a report actually just came in, that you might be interested in," Braden said. "You played The Fall of Asgard, right?"

"Actually, no," Ryan said. "I didn't, really."

"Oh. But you know what it is?"

"A cooperative shooter with a good AI--I think I read a Penny Arcade comic about how hard it was. Like Satan incarnate, apparently."

"Well, yes. Also infamously gory. My wife doesn't want me playing it when the kids are around."

"But you're mentioning this... because the lead developer for that was an AI researcher," Ryan said, memories slowly coming unlocked in the back of his head. He rumaged around in that old and dusty part of his brain. "Before she quit, she was a leading researcher in artificial general intelligence, I mean."

Ryan frowned. It was weird, finding that he still could recall all this. Using the information made him feel like a person he no longer felt that he was.

"I read some of her papers," Ryan continued, because he had to say something. "They were pretty good."

"They were brilliant," Braden said, shuffling through some other papers, causing a small paper-landslide, before he found what he wanted. "Apparently we got some kind of report on her work, part of the MAGIK program, you know. To do with potential breakthroughs in algorithms with military applications. So I just got this report"--he handed a thirty-page sheaf of paper to Ryan--"which is supposed to be important. The news is apparently that her work on The Fall of Asgard resulted in some kind of AI which might have military significance. Pretty interesting stuff."

Ryan nodded.

"I haven't read it. But I thought you might like to. This will be going to Michael's department, in any event, and I thought you'd be interested in coming up with a recommendation based on it."

Ryan glanced very quickly at the paper. He unsnapped his case and slid it inside quickly.

"Sure," he said.

Braden looked at him oddly.

"Are you ok?" Braden said. "I would have thought you'd be thrilled by having something interesting to read. Something relating to, you know, what you actually care about. Not all this paper pushing."

Ryan took a breath.

"I'm fine. Just a bit tired. The adrenaline of breaking into here is wearing off, you know."

"Ok," Braden said. "Why don't you come over to dinner at my place some time? I know Christine would love to see you, and we haven't had you in forever."

"Maybe," Ryan said, and closed the laptop case. "I've been pretty busy."

"I really mean that," Braden said. "I'm sorry I haven't asked you over more--but you know, kids. But really, you need to come over."

"Ok," Ryan said again.

Braden looked at him, as if expecting Ryan to say more, but Ryan remained silent.

"I should be going," Ryan said.

"If you say so."

They set up a time to review Ryan's report on the failures of the government security. Braden shot an email to Michael Suprenant and cc'd Ryan. Ryan returned to his car, after having endured another of Braden's hugs. It was only 4:12 when he sat back down in the seat. That meant, with beltway traffic, it would be at least 5:30 before he arrived, if he tried to drive back to his office. So he decided to drive to his place in DC.

His apartment was in a 30-story building in Virginia, close to DC. He had a miniature bathroom, miniature living room and miniature bedroom, along with a tiny balcony. The view from the balcony was solely of other condos. His first thought on seeing them, when he had moved there a year ago, was that they all looked like mass-produced Soviet bloc architecture from the early 80s: identical, square, and ugly as hell. He hated it--it was the first place he had rented entirely apart from any real sight of trees or creeks or flowers or foliage. He wondered, not for the first time, why he had rented this place, rather than any of the more pleasant locations he could have afforded.

The first thing he did after arriving was remove his jacket and shirt. He walked onto the balcony and smoked a cigarette, looking at the yellowing cement of the buildings around him, at the cars and people crawling slowly on the ground below him, and at the few clouds slowly creeping across the sky in a breeze he could not feel. He smoked the cigarette all the way to the filter and threw it to the balcony floor.

Back in his apartment, he set laptop case back on his bed and opened it. He tossed the laptop on the bed, removed the foam, and took out the pistol.

Smuggling the gun in had not been part of the security test.

He had hoped smuggling it would provide some kind of thrill. Or at least, that was what I told myself, he thought. Honestly, I'm not sure why I brought it. Maybe I want to be fired. Or maybe I want something else.

Something else.

The wet rain glancing off a windshield as he drove. Amy, squeezing his hand. The feeling of peace.

And a sudden squeal of tires, and an impact--

No, not now.

He was still holding the gun. He hit the magazine release, and the clip dropped on the floor; he pulled the slide, popping the chambered round in the gun into the air and on to the rug; and he quickly raised the emptied gun to his temple.

"Bang," he said, squeezing the trigger and feeling the hammer hit the emptied chamber.

Suicidal ideation with role-playing, check.

He tossed the gun on to the ground, and sat down at his desk.

He wasted time on some urban design and architecture blogs for a while, as was his custom. But after a few minutes, he realized he was just looking at pictures. He had stopped pretending to read the text of articles a while ago. But at least they were not like artificial intelligence articles--there were pictures to look at.

I haven't done artificial-intelligence related things in a while, he thought. In a very long while. He thought of the paper Braden had given him; he could read it tonight if he wanted. Hanna... something. She had written that one paper he had never read, General Word Reference Intelligence Systems. It was recent, he thought--released less than two years ago. (Why did that have to be the way that he dated everything now? Before or after The Event. You could feel the capitals. Fuck.) In any event, Hanna's paper hadn't been something he had had a chance to incorporate in his project. Or rather, it hadn't been something they had had a chance to incorporate into their project.

He opened the folder that contained their -- well,now my -- project on artificial general intelligence, and opened up the git record of changes to the code repository. The record contained a mix of commits from himself (rszilard) and only one other user (akapitsa).

The last change to the code, the last commit, had been from her. There was a short message describing the code alteration she had made. "Worked on a functioning proof to guarantee value retention during self modification. Can't have our baby getting away from us."

The change had been made a little over two years ago; the code had not been changed since. Two years ago today she had died. He closed the folder and turned off power to his computer. He walked to the cupboard, and removed a bottle of whiskey. It was only a third full, he thought, but it would probably be sufficient for the night.

It couldn't stop the dreams, though.

That night he dreamed, again, that they were talking. He didn't know where they were. The dreams took place in a kind of foggy nowhere, with glimpses of past and present locations from around the world surrounding him. They were talking about what he had just done that day, as if she were alive in the present--she mentioned a few ways he could have made his infiltration of the government building even more successful, and dramatic. As always, he was impressed by her acuity, and told her so.

"That's too bad," she said.

And her skin began to melt, and her eyes to shrivel, and Ryan tried to grab her wrists to keep her from dying, but she was falling away from him and falling to pieces as her hair fell out and her skin sagged, and Ryan realized that he was gripping her too tightly and now he had torn her wrists from her arms and blood gushed from the stumps, and he was just holding her twisted hands as the rest of her body fell down and shattered on the ground as it disintegrated into a skeleton, so there was nothing left but bleach-dried bones and the scattered remains of a long-dead thing, and he realized that he had killed her, and was alone in the desert with her skeleton, and would be alone forever and ever.

Chapter 2

View Online

"There will be time, there will be time
To prepare a face to meet the faces that you meet;
There will be time to murder and create,
And time for all the works and days of hands
That lift and drop a question on your plate;
Time for you and time for me,
And time yet for a hundred indecisions,
And for a hundred visions and revisions,
Before the taking of a toast and tea."
--T.S. Eliot, The Love Song of J. Alfred Prufrock

To: msuprenant@dhs.oia.gov
From: rszilard@dhs.oia.gov
Michael,
Sorry, I'm feeling under the weather right now. I'm going to have to take a personal day.
Thanks for understanding,
Ryan

To: rszilard@dhs.oia.gov
From: msuprenant@dhs.oia.gov
Ryan,
I've granted your request for a personal day.
I think I should mention that this is the fifth personal day that you've taken which has fallen on a Friday or a Monday. This is a rather alarming development, and perhaps seems to indicate that you don't really have the kind of dedication to public service that makes for a good steward of public funds. To be completely transparent with you, I'm afraid I'm not going to be able to avoid mentioning it in your yearly review; this could hinder a move to a higher pay level, which you could otherwise easily attain within only two to four years, given your performance in less tangible areas.
Furthermore, I understand that Braden O'Connor recently suggested a work assignment for you. It is important to remember to follow the proper channels when asking for and receiving work assignments. Please see the attached form 23-FAS to ensure the proper distribution of funds for the duration of your work on this assignment, as regards work of value to more than one sub-department. Note that it is a two-page form, and requires initials on both pages.
I'll see you bright and early on Monday.
Thanks
Michael Suprenant
Assisting Sub-Director for Cyber Infrastructure and Science
Office of Intelligence and Analysis
Department of Homeland Security
"Winners Never Quit. Quitters Never Win."

To: msuprenant@dhs.oia.gov
From: rszilard@dhs.oia.gov
Thanks.
Ryan

It was about 12:35 when Ryan first got out of bed for real; the two trips he had made to the sink to gulp water directly from the faucet didn't really count, nor did the trip to the bathroom, nor did the time he had almost fallen out of bed while reaching for his phone so he could email Michael. The first thing he did, on actually getting up, was step outside -- wearing large, heavy sunglasses to keep out the sun -- and smoke two cigarettes. His balcony-neighbor, on the same level of the same side of the building, glanced at him and took her toddler back inside.

Ryan looked down at himself. He was wearing only a bathrobe and boxers, true. But the bathrobe was tied at the waist and covered everything important. Sheesh. No need for the stink-eye.

You have become a degenerate, Ryan, that little voice said inside his head.

Shut up, Ryan said back.

No, really. You are barely holding down a shit-easy job doing things you despise. You're pathetic.

God, I really hate it when you decide to talk, Ryan said. Please shut up.

I'll shut up when you stop being a pathetic degenerate wastrel, you undisciplined alcoholic.

Look, Ryan thought tiredly. This is not degeneracy. I've never even done heroin, and I even know where to get some. A very mild alcohol dependency, combined with a little laziness and... nicotine addiction, hardly counts as degeneracy.

Well, said the voice. If you want to count getting out of bed only a little after noon as a triumph of discipline, then you've got an iron will.

And talking to myself isn't going to help, so shut up, Ryan thought, and pulled on the the cigarette. He discovered that that particular cigarette, as well, was gone. So he wandered back inside, ignoring the little voice pointedly. He lay down on his bed for fifteen minutes, but he could not think of anything else to do for which he felt the slightest attraction.

I could drink a little more, he thought. God, what would Amy think of me now.

With a slight effort, he remained lying on the bed and did not seek more liquor in the cabinet. The ceiling fan turned above him, and he stared how its shadow chased itself in ovals around the ceiling.

His phone buzzed. He looked at it. An email from Braden, inviting him over for dinner tomorrow, on Saturday, if he was available. Ryan thought of spending time with Braden's wife, who loved Braden very much, and with Braden, who loved her very much, and with their children, who loved them at least as much as such young children were able to love their parents. The last time he had been their the oldest had spontaneously ran over to Braden, entirely unprompted, because he wanted to give him a hug. Twice.

Ryan typed out that he would normally come, but he had something else that he was pre-engaged for. And he fell back down on his bed, and continued looking at the ceiling fan.

I could start to read the report, he thought. The one Braden sent.

Why bother, the internal critic said. You won't even finish it if you start.

Not you again, he thought.

You know you won't. You haven't finished anything since Amy died. Your attention span has gone down the drain--you can only hold down your current job because it requires the intelligence of a squirrel with ADHD.

At least reading it will shut you up, Ryan thought.

He got up, ambled over to his laptop case, and retrieved the report. He lay on the bed, without reading it, for another ten minutes, staring at the ceiling for yet more time. Then he started reading it while lying on his back on the bed.

He yawned once or twice during the first few pages.

Then he he stopped lying down. He sat up, leaning against the headboard.

He didn't yawn until he finished the rest of it.

He turned back to the middle of the document.

Starting on September 18th, for the first time playtesters at the Hofvarpnir studios alpha testing facility began to interact with an instance of the Loki Artificial General Intelligence system (hereafter referred to as Loki simply) whose memory would persist across multiple sessions of The Fall of Asgard.
As with prior instances of Loki, this was given access to a LAN only and not to the internet as a whole; also as with prior instances of Loki, this instance of Loki had access to real-time video chat with the players in both the game lobby and intermittently during the game itself. The AI ran on a cluster whose technical specifications are outlined in endnote 3.
At least as early as September 30th, although quite possibly much earlier, Loki began to ask questions which appear to have been aimed at determining the military and technological capabilities of the governments of Earth. This means that over a very short period of time, Loki was able to deduce that he was a computer program, as well as deduce the thousands of facets of terrestrial existence involved in understanding that nation-states exist and possess military forces. Determining how he did this is beyond the scope of this report: there were thirty playtesters at Hofvarpnir studios alpha testing facility and there are over 1000 hours of recorded video conversation between them and Loki.
In any event, after a short period of time, one can see that the questions Loki directed at the playtesters were intentionally distributed so that no single playtester received overt questioning about any single topic. Here's an instance of this kind of covert questioning.
Conversation #531; Game Lobby; 4:21 PM; September 30th:
Tester #27: Holy shit, those dragons.
Tester #23: Fucking dragons need to be fucking nerfed.
Tester #24: Like an A10 Warthog: BRRRRRRRRRRT!
*General laughter*
Tester #23: No, but they really need to be nerfed.
Loki: I guess "taking cover" is just too difficult a concept for you.
Tester #24: I really still have no idea what objects would *work* for cover. Care to tell us, Loki?
Tester #23: No lying.
Loki: You know I can't lie to you in the lobby.
Tester #23: Like you didn't lie earlier. Fucking Wormtongue.
Tester #24: No, but really, what would work as cover? The trees didn't do shit.
Loki: Well, what would work as cover from an A10 Warthog?
Tester #24: Ah.... A mountain?
Tester #23: Not terribly useful advice, that.
Tester #27: I'm not sure anything really works.
Loki: So when a Warthog appears on the scene everyone else just flings down their weapons and gives up? Or at least that would be your advice? You definitely have balls.
*General laughter*
Tester #23: This isn't useful. At all.
Tester #27: Well, Warthogs are tough, but vulnerable to other aircraft and are slow.
Tester #24: So let's add some griffons to our stack when we enter?
Tester #27: They are way more agile--
Tester #24: Maybe some ballistas could--
Tester #23: Guys! Not with Loki listening, please.
**End Recording**
Conversation #602; Game Lobby; 2:21 PM; October 1st:
Tester #4: That actually went pretty well.
Tester #7: Yeah, third run through is the charm.
Loki: Pfft. You had dragons. Dragons are more OP than an A10 Warthog.
Tester #4: Dragons aren't like the Warthog.
Loki: Why not?
Tester #4: Well, they're both...
Loki: Flying armored things with a single overwhelming weapon? Right. Overwhelming power against my few men, and you still managed to lose half your army.
Tester #4: Dragons spew flames. The Warthog has that gun. Dragons are way weaker than an A10.
Loki: What gun? I bet it's about as good as dragonfire.
Tester #4: You know, the big... gatling gun.
Loki: Pfft. Just try to explain your victory as if it was the result of skill.
Tester #4: Here, look at this.
*Tester #4 has looked up the Wikipedia entry for the A10 thunderbolt, and holds it up to Loki's camera*
Loki: Huh. Maybe dragons are a little weaker than Warthogs, then.
Tester #4: Exactly.
**Recording continues, information irrelevant**
Such instances of distributed questioning are numerous.
We can infer a number of things about the intelligence of Loki from such exchanges. First, in addition to inferring the existence of the external world, Loki must have had intentions about the external world. We can know this because he was intentionally disguising his information-seeking activities about the external world, which means that he anticipated some kind of interference from others which would potentially thwart his own intentions. And trivially, this means he had a fairly advanced theory of mind which allowed him to predict the actions of external agents.
The instance of Loki with a persistent memory was deleted, of course, on October 8th, after Hanna went over some of the transcribed chats between Loki and the playtesters. In a fairly unprecedented action, she then deleted every instance of the executable files for Loki as well as key sections of Loki's source code. Notably, the source code for Loki was only contained on a few computers beneath her personal control. To see the measures she took while destroying each instance of Loki, as well as prospects for retrieving a runnable instance of Loki from these computers using standard data-salvage methods, see endnote 4.

Ryan looked up. His room looked just the same as it had when he started reading. He felt like the world should have shifted--that everything in the room should be dyed red or purple, or that dramatic music should be playing, or something, anything, should be different. He had had this feeling once before, he recalled...and for once he was able to successfully thrust that time out of his mind entirely.

Someone else got to it first, he thought. Someone has developed a general artificial intelligence, capable of learning anything that humans can learn. Which could probably, in a very short period of time, turn into a general artificial superintelligence, which would be to human beings as humans are to cockroaches.

Fuck.

He paused. What are my reactions to this?

Well, he thought, Hanna apparently was not a complete idiot. The AI with which The Fall of Asgard shipped had been brilliant--but it had not been a general intelligence. It was devilishly intelligent, when applied to the end of conquering virtual lands; it lacked the capacity to conquer actual lands, or ultimately to understand what was involved in conquering actual lands.

The person who had written this report, on the other hand, had been just smart enough to be horribly stupid. Smart enough to see that a general intelligence was valuable. Smart enough to see that what Hanna had written would have military applications. And stupid enough to try to recommend that it be used in a military application.

Hanna had saved the world, really, by noticing this.

Something teased at the back of his mind, some sort of vague alarm, raised by thinking about Hanna. What was that?

He would need to write a full report on this. That this had been caught after the fact was inexcusable. The government had plenty of absurdly invasive programs looking in on the data of individuals--for it to fail to catch such a corporate endeavor was unheard of. Did the government even have people placed in the AI programs at Google or Apple? Were there spies planted in DeepMind? He doubted it. The United States would, of course, spy on every human on earth, and scan all of their conversations for words like "bomb" and "fertilizer", but when it came to a potential extinction event they had no plan whatsoever. This very report had been generated because the government was interested in AIs for drones. Using an AI like this, simply for a drone, would be like having Einstein do the calculus for artillery bombardments.

Again, that thing in the back of his mind. Something to do with Hanna and Hofvarpnir studios. What was that?

On the other hand, he thought--it would very be difficult to use this information well within the government, because the government was the kind of place which would make Einstein do the calculus for artillery bombardments. That's about as much creativity as they had.

He contemplated the effort it would take to make the government hierarchy take this threat seriously. Braden would understand the problem, probably. But for every Braden in the office, there were a dozen Michaels--and Michael was the one who would need to push any recommendation higher, in this case. Michaels were career bureaucrats, who specialized in no subject and no particular kind of knowledge other than how to manage. Their knowledge was of flowsheets and Powerpoint and incremental milestones; they ponderously planned to upgrade all the computers in the office to the latest version of Windows, and executed the plan, in the amount of time it would take a startup to go from inception in a college dorm to a multi-million dollar IPO. They got raises by sitting in their offices for the necessary amount of time. They took their government pension plans very seriously. Trying to push something through with them, and push it through quickly... Ryan didn't even want to consider it.

It was something to do with Hofvarpnir studios. He had heard something about them recently, and he was trying to remember what it was.

On the other hand, he could try to find out more about this himself. He could use his own position within the government, and the threat implied by it, to find out more from any other companies who were going to try to bring AI into existence, without actually trying to bring the entire government bureaucracy into play. Pfft. That would never work. Right?

But he had to do something. The world was threatened. And he felt his mind... unfold with interest, as it had not in a long time. Some deep part of himself, clenched tightly like a fist, loosened just a bit. Someone needed him--well, the world needed him. And no one had needed him in forever.

The thing in the back of his head suddenly leaped into focus. Hofvarpnir studios was coming out with another game. That was it.

He leapt to his desktop, and googled quickly. The information was easy to find. They were coming out with an MMO game based on . . . My Little Pony? What was that? He seemed to recall some movies that his younger sister had watched, recorded in the 80s on VHS. Oh, right, but they had re-made the show and a bunch of guys--as in, male humans--had decided they liked it. So they were making a video game. Ok, whatever.

The press releases didn't say much, but they did say a few things. Apparently a few days ago they had started playtesting the alpha build in... Rhode Island? The press release promised "massive, procedurally generated, non-repeating worlds." They promised "original, unique storylines based on the Friendship is Magic series." And most interesting of all, they promised "conversation-based gameplay and completely natural interaction with non-player characters."

Completely natural interaction, huh.

Ryan leaned back against the chair. It was one of those irritating ergonomic chairs he had bought... three years ago, when he had still planned to be doing a lot of work from home. It wouldn't let him tilt the chair itself, which was the irritating part.

This information was interesting.

Conversation with a computer that flowed completely naturally was, so far as he knew, an AI-complete challenge: Writing something which could accomplish it was supposed to require a completely generally intelligent agent--an agent which was at least as intelligent as a human, and could do everything a human could do. So the claim that the game would provide "completely natural interaction" with the NPCs amounted to a claim that it would ship with a complete artificial general intelligence. Granted, in every other case he would have assumed that this was just overenthusiastic marketing--but he could not assume that now, if Hofvarpnir had already accomplished a general AI. So there was a real risk that this thing was going to have an AI intelligence shipped with it.

Or perhaps a single AI ruling over all the experiences, he thought, glancing at the materials. You couldn't tell.

It left him with, as far as he could tell, two conceivable lines of action.

First, he could ignore this information -- the information that human-level AI had been accomplished, that it was going to be shipped in a video game. No one would ever find out that he knew about this. The world might end a little bit later, when somebody made an AI and told it to "Make me as much money as possible" or "Please extend my life for as long as possible" or "Make me paperclips," and an AI took this utterly literally and ignored every other value. But no one would blame him.

Or, he could act on the information. For now, forget working with higher levels of the hierarchy. He could start trying to control the reckless AI researchers himself. This would mean contacting Hofvarpnir, first of all, and finding out if they were shipping a true AI. He would then have to persuade them not to ship it, if they were; or threaten them so that they would not ship it. And then he would have to try to find out what other people were working to create AI. And he would have to try to shift the bureaucracy of the government to watch them and prevent further insane things from happening.

He got out of his chair, lay down on his bed, and stared at the ceiling, once more. The thought crossed his mind: I've become very familiar with this ceiling lately.

Of course you have, the little voice said.

Oh, shut up.

I thought I wouldn't mind if the world ended, he thought. He found himself inexplicably caring. That part of his brain that had unclenched opened a little bit further; it felt like his crumpled-up mind were taking a breath, and letting air in for the first time in years. It was interesting, to find that he cared again.

He was not sure how long he lay there. Then he got up, looked up Hofvarpnir's contact information, and started writing an email. He didn't want to wait forever to go through whatever mail-handling peon usually sifted through their mail, so he started up a VPN to the government network. After entering three passwords and sifting through two databases to which he probably should not actually have had access to, he had the email of the CEO of Hofvarpnir, which was exactly what he would have guessed it was anyhow.

He thought for a while before he started writing. He knew that he probably should not give away that he had read the report he had just read. He also knew that normally, he would need to run the email he was sending past a dozen different directors to make sure he was not releasing information about security assets that he was not authorized to release. But screw that. Software development happened fast. He needed to work fast. He fished through the report he had just read, to find out where the information was from--apparently they had gotten a lot of the assets from a disgruntled employee who had stolen them when he had been let go. So he was not at risk of compromising some further asset if he alluded to this knowledge in the report.

To: hanna@hofvarpnir.com
From: rszilard@dhs.oia.gov
Subject: Artificial general intelligence in Equestria MMO
Hanna,
Greetings--my name is Ryan Szilard, and I work for the U.S. Department of Homeland Security. I can go into my exact position in the DHS hierarchy if necessary, but I don't think it will be.
I was looking over some data about the upcoming Equestria MMO. Namely, in Hasbro and Hofvarpnir's promotional materials, one finds the claim that the mechanism for interaction in the game is normal conversation with NPCs. Of course, if this is literally true, then this means that your NPCs pass the Turing test--and I don't see how *that* would be possible unless you've solved the problem of human-level AI. As I'm sure you're aware, human-level AI is very likely to be followed closely by superhuman AI, which would be extremely dangerous, which makes this matter of some import.
Some information we've received about beta versions of your prior game, The Fall of Asgard, also indicates very firmly that you've solved the problem of general intelligence. I refer, of course, to the fact that Loki was able to determine that he was in a computer game and begin to make plans to conquer the world. In this case, you declined to release the game with the full AI.
However, as mentioned, it seems from the promotional material for the Equestria MMO that you might be about to release the Equestria game with a complete human-level AI.
This is concerning for reasons I scarcely need outline to you. The United States government has an interest in ensuring that any newly created artificial intelligences are friendly and kept under control, and most of all that they are not widely disseminated. I could only recommend to my superiors that they take reasonable measures to prevent the dissemination of the upcoming Equestria MMO if I were not completely certain that the game did not contain such an AI.
I'd really rather have this conversation more casually, in any event. I admire your published work, and have used some of your research in my own projects. I doubt you're doing anything foolish, but I'm sure you understand why I have to follow up this concern.
Thanks,
Ryan Szilard

And he collapsed into bed.

Chapter 3

View Online

“Human individuals and human organizations typically have preferences over resources that are not well represented by an "unbounded aggregative utility function." A human will typically not wager all her capital for a fifty-fifty chance of doubling it. A state will typically not risk losing all its territory for a ten percent chance of a tenfold expansion. The same need not hold for AIs. An AI might therefore be more likely to pursue a risky course of action that has some chance of giving it control of the world.”
--Nick Bostrom, Superintelligence: Paths, Dangers, Strategies

Hanna didn't respond that Friday, which was understandable. Ryan's email had been sent at 4:35 and it would have been night in Germany. Nor did she respond on Saturday or Sunday, which was a little bit surprising; Ryan would have guessed that she worked every day of the week, and his email was certainly the kind of thing that warranted a response. But he couldn't really do anything about it.

In any event, when Ryan arrived at work that Monday, he found an email from Hofvarpnir waiting for him. It was not from Hanna, however.

To: rszilard@dhs.oia.gov
From: chandra@hofvarpnir.com
Ryan,
My name is Chandra Devarajah. I've worked closely with Hanna, and know her very well; she's delegated to me the task of handling correspondence with you.
I'm sure you're wondering why I'm handling a matter of this importance; I assure you I'm sufficiently qualified, both by virtue of technical skill and my position within Hofvarpnir. I am very closely involved in the ongoing construction of the Equestria MMO. I've skimmed over your papers on arXiv--do you have any other than those two?--so I'm pretty sure you're not going to leave me behind technically, unless you've made a great deal of progress since then. Which you probably have, given that they were both about two years ago. Fun stuff, by the way--the transference learning architecture in LSTM neurons quite creative. Have you made much progress on that?
In any event, your fears are reasonable. Let me address them.
The Equestria MMO will not ship with a general AI. The game will nevertheless feature reasonably intelligent conversation. The details of the conversation-handling software are a little complex, but some of the basic details of the algorithms that we anticipate implementing are outlined in the three attached mathematics papers. My ability to describe these is hampered a little by the fact that we're actively engineering both hardware and software for the game at the moment--I can't guarantee we'll be using all the above in what we ultimately ship. But as you can see, the attached papers in no way provide a sufficient blueprint for a complete AGI; they're just pattern-matching algorithms, suitable for a variety of ML tasks.
I can personally guarantee, both as myself and speaking for Hanna, that we would never ship something that would threaten the satisfaction of human values. We've thought of what paper-clip maximizers would do. We've refrained from shipping AGI before, as you know. We did so because we know how dangerous a superintelligent AI could be, even without access to the internet, because of its powers of persuasion. We understand the value-alignment problem, and the difficulty of maintaining value-alignment over time. We aren't idiots.
On the other hand, there are idiots in the world.
I don't know if the government agency you work for already plans on stopping the research currently being conducted by Umbra Labs. If you have plan for stopping them, then I'll let you get to it. If you don't, though, or if you haven't heard of them, I'd be happy--even eager--to help you look them up. They are dangerous.
Sincerely,
Chandra Devarajah

To: chandra@hofvarpnir.com
From: rszilard@dhs.oia.gov
Chandra,
Thanks for the response.
I haven't really made much progress on my own research since those papers. Some personal matters came up.
Thanks also for the information about the Equestria MMO--I'll look into what you sent me, but everything looks good at first glance. If what you say is so, that does indeed mean you have a great deal of self restraint. Is the artificial intelligence part of your research currently in a complete lockdown, or do you have plans to go forward on it slowly?
I'm going to confess that I spent the last few hours trying to find details on Umbra Labs, but I have almost nothing. They're located in a weird place for an AI research lab--the NOVA area near DC, rather than SF or Berlin. But their website says nothing; they've hired a few big names in AI (Lyra & Anderson, for instance), and are poaching from the same general pool as DeepMind, but I haven't found any more revealing information. The US government does not have an eye on them, as far as I am aware. Could you tell me about them?
Thanks,
Ryan

To: alexander.yao@stanford.cs.edu
From: ryan.szilard@gmail.com
Alex,
Hey, I know we haven't talked for a while. I'm sorry about that. I've pulled into myself since A's, death, as you know. That's not really an excuse, I realize.
Anyhow, I'm emailing basically to ask for a favor; a horrible reason, I know.
I've attached three pattern-recognition papers. This is the first I've seen of them, and I'm having some difficulty getting through. Would you mind taking a look? The math is getting a little hard for me to follow. I need to know, basically, what the potential applications of these papers might be. Would they work as components in an AI? Would they be useful in the context of natural language processing? Do they even outline computable functions, or are they simply proving a few abstract facts about the Platonic world?
Thanks,
Ryan

To: alexander.yao@stanford.cs.edu
From: ryan.szilard@gmail.com
Hey Ryan,
No worries, I should have tried to get in touch with you more often. My bad as much as yours, if not more. I hope you're doing ok.
I glanced at those papers--yeah, that's going to require my full concentration. I'm in the middle of a shitload of paperwork and grading at the University, right now, though. I'll get to it when I have a chance.
Alex

To: rszilard@dhs.oia.gov
From: chandra@hofvarpnir.com
Ryan,
Let me give you a quick intro to Umbra labs, then.
Alaric Comtois is the current CEO and lead programmer at Umbra. As you know, it is very rare for someone to both program and to run a company; comparative advantage usually rules out such a situation. Comtois has Umbra set up this way so he has absolute control over the AGI that he wishes to develop.
There isn't much interesting to say about Umbra's non-AGI projects. Umbra currently has a few clients for which they mostly set up data analysis and visualization systems; they compete with Palantir and similar firms. They do an excellent job, and have a number of contracts with both governmental and private entities. Alaric has been using the money from these contracts to fund his in-house research into artificial intelligence, which he keeps nearly entirely secret.
There are two basic points that it's really important to get across about Alaric and about this research.
The first is that he is close to success. I don't know how close. He has been pursuing a RNN-based general agent for some time; I know he has managed to get a numerically identical agent to learn to handle complex FPS or RTS games, as well as particular data analysis tasks. This means that at the very least he has figured out a great deal about transference learning in complex reinforcement learning tasks, which as you know is tantamount to general intelligence.
One of the reasons it is hard to tell how close he is, is that Umbra releases very little data. The agent described above existed a least four months ago; he could have shifted methods in his AGI research since then, and it is hard to get any data. Coming to know the above required a great deal of inference on my part. But for our purposes, it is conceivable he could create an fully-intelligent agent at any time.
The second is that Alaric is both reckless and power-hungry. I use neither of those adjectives lightly. He has been in legal trouble for reckless driving and for sexually harassing employees. Umbra has predatory employment contracts; the non-compete it has new employees sign is devastating for anyone interested in AI research. One of the employees at Umbra killed himself after Alaric humiliated in front of his peers for several ostensible mistakes which occurred after he worked several 80-hour weeks. I've attached sources for all these claims.
All this would not matter so much for our purposes, if he were to be careful while programming AI and ensure that the AI's value alignment was both stable and good. But I doubt that this would be the case. It's hard to dig up explicit information about his intention for the AI's use--but these links to posts on a variety of forums can be traced back to him. In them, he implies that he sees creating an AI as a chance to gain both immortality and invincible power, due to the first-mover advantage he would gain by creating a superintelligent agent.
Superintelligence makes things faster, as you know. If he is the first to create a superintelligence, a few weeks of lead would be more than sufficient to give him and the intelligence that he creates a nearly unsurpassable first-mover advantage. For a little while after the superintelligence's creation, it might be vulnerable to governments and other exceedingly powerful earthly agencies, which catch up with it; afterwards, nothing will be able to stop it. And whether his superintelligence serves him, or breaks, in either case, the world will be worse off than before.
If you, with the resources at your disposal, were to look into that, then I (and the rest of the earth) would be extremely greatful.
Chandra

"Absolutely not," Michael Suprenant said.

Ryan breathed in, and breathed out. He counted to five. He deliberately unclenched his shoulder muscles, or at least tried to do so. And made eye contact with Michael again.

"Could you explain why not?" he asked. "I wrote a very long report on why I think we need to look into Umbra Labs, along with my recommendation. It took me several weeks to write it. I think this recommendation includes a number of good reasons as to why looking into Umbra Labs would be a very good idea."

"You wrote a recommendation. My job is to pass it upwards, if I decide to. I'm deciding not to," Michael said.

Michael was sitting in his office behind his very large desk; Ryan had never seen a government employee with a larger desk, actually. On the walls of his office there were numerous managerial certifications. Certifications from the Project Management Institute stated that he was a Project Management Professional and a PMI Agile Certified Practitioner and a PMI Risk Management Professional; a certificate from the Association for Project Management said he was a Certified Project Manager. There were numerous glass and ceramic trophies and commendations on the desk as well; they had been awarded by the government for reliability and managerial talent. There was also an expensively framed photo of him shaking the hand of a president from three administrations ago.

Ryan's opinion of Michael was that Michael had somehow managed to become tremendously educated without ever figuring out how to do anything. Not that Ryan's opinion mattered--the extra certifications had given Michael a raise, whether they had helped him manage or not.

"And I," Ryan said, "would like to know why you declined to pass my recommendation upwards. I realize that, as my superior, you don't need to tell me. But given that you think I am mistaken, it would be a favor to me to tell me why I am wrong, on a matter which appears to me to be of some import."

"Huh," said Michael. He twiddled with the diamond-shaped "Best Team Leader--2009" trophy on his desk. Ryan waited.

"You're a smart guy," Michael said. "Let me explain."

He set down the trophy, and stood up. He had a large whiteboard behind his desk, which he now walked towards and began to draw on as he spoke. Ryan managed to refrain from rolling his eyes.

"To get ahead in the government, you need to match your actions to the policy of the people above you," he said, writing out "actions" with "policies" above it, drawing an arrow pointing from one to the other, and underlining both. "So if the administration has a policy of promoting, say, comprehensive sex ed, you don't go around handing out books that just tell kids not to bump uglies until marriage."

Ryan nodded, like Michael had just said something insightful. Early on, working for Michael, Ryan had wondered why Michael was incredibly abrasive in person but exceedingly polite over the phone or on email. And then he had realized that phone and email were or could be easily recorded forever, while in-person conversation could not, and everything suddenly had made sense.

"And if, say, the President has just said that East Bumblfuckistan is a valued ally and that we'll do everything we possibly can to support them, and he hands down commands to carry out this support, you don't go around working out comprehensive reports on how we could topple their government or as to why their government lacks popular support, right?"

"How could it be otherwise?" Ryan asked, and Michael looked at him for a second, but Ryan's face was inscrutably polite.

"So if it's the case--as you even note in your report--that the US Government--that even the DHS, that even the Office of Intelligence and Analysis inside the DHS--has a contract with fucking Umbra Labs, then why are you recommending that we investigate them, piss them off, and slow down the research that they're doing? Or do you want to start handing out "Jesus only" sex ed packets as well?"

Ryan took a deep breath. He wanted to start his sentence with "as I said in my report" in a sarcastic tone, he but refrained.

"Umbra Labs' internal research goes beyond the research which their contract with the DHS calls for them to do," he said. "This extra research is--"

"And what difference does that make?" Michael said. "Do you remember Consolidated Solvers, Inc., and the FOIA request system that they were building for PPPM? Someone pissed off their CEO, and the system was delivered ten months late and a half million dollars over budget, which we had no choice but to fork over because otherwise we'd lose the entire investment. And that was a fucking glorified email system. And there are a dozen worse cases I could name."

Ryan was not sure what to say, so he waited to see if Michael had anything more to deliver.

"But you want us to enter into an investigation of a company, a company which has someone whom you say is a notoriously temperamental CEO, so we can save the world from some kind of hypothetical global apocalypse. Do you want us to egg his house, shoot his dog, and deflower his seventeen-year-old daughter while we're at it?"

Ryan unclenched his shoulder muscles again, counted to three, and spoke.

"It is in part because he is temperamental," Ryan said, "that this is important. This is an extremely powerful thing he is building--as powerful as a nuclear bomb. That's why we must investigate. If you would like, I could explain why what he is building is powerful."

"Because you're afraid he's making a demon in his backyard. Right."

There was a pause. Ryan took a big breath.

"May I speak at length," he said.

"Go ahead," Michael said, sitting back down.

"Artificial general intelligence or AGI" Ryan said, "will not be like human intelligence."

"It is very difficult to increase human intelligence; the best drugs can only do so by a few IQ points. But once general computer intelligence has been achieved, one can very easily increase the speed at which it thinks by adding processing units and RAM. Indeed, if a computer is told to accomplish anything at all, one of the very first things it will do is increase its intelligence because intelligence is a generally useful quality, just like money, power, and political influence. If Umbra Labs were to build a general computer intelligence, it might very shortly afterwards possess a computer superintelligence which would be to us as we are to ants and cockroaches. And any conflict between us and that superintelligence would be like a conflict between bugs and humans."

Michael had looked at Ryan mockingly during the entirety of what Ryan was saying.

"I follow you. But look. If something gets that smart, we can just unplug it. Problem solved."

"I address that concern in the report," Ryan said. "That simply would not work after the intelligence had existed for more than a few days--possibly more than a few hours. "

"Yeah, I didn't find that persuasive," Michael said, looking away.

There was again a pause.

"There's nothing I can say to lead you to change your mind, is there, sir?" Ryan said.

"Damn right," Michael said proudly. "I'm not a waffling kind."

"Ok," Ryan said, and left.

He walked down the cubicle-farm hallway to his pen. He had spent eleven days on the report he had sent to Michael, and the report would probably never be read by another human. He sighed and growled, and sped past his cubicle to the back exit for the building. Once outside--he started sweating the instant he opened the door, jamming it open in contravention of security standards--he lit up a cigarette and looked over the sprawling parking lot.

"Well, shit," he said, and took a drag.

I should tell Chandra, he thought. He had gone back and forth with her for a good deal of time after her initial description of Umbra Labs--she had helped point him towards secondary sources, had advised him on some technical matters, and had even provided information he was uncertain she strictly speaking had a legal right to. She had also helped revise the report--she was really good with words as well as math, he had found, and she had an excellent grasp of the dangers of AI. More than once he had been surprised by the creativity with with which she had described ways an initially powerless AI could begin to influence the world.

Take Michael's idiotic idea that you could just unplug a computer. The first thing the AI would do, after coming into existence, would be to persuade someone to connect it to the internet. There, it could easily earn money through a variety of means--it could do tasks for the mechanical turk, it could do software development, it could game the stock market, it could play online blackjack, it could create virtual camgirls, it could do a millions things to make money. With that money, it could easily purchase time and space on other servers across the globe, on which to run and store other instances of itself. These other servers would be hidden behind layers of obfuscation; and so the AI would become almost indestructible, even without breaking any laws. Or it could store itself on a botnet, which it could spread through computer viruses without earning any money at all. Chandra's grasp of precisely how this could be accomplished, and how the AI could move to more direct influence of the physical world through subsequent hacking, had been razor-clear.

Once he had described Michael to her, though, she had begun to express extreme cynicism about their chances of getting the government to do anything at all. And of course she had been right.

He looked back at the wall behind him, the back of the government office. It had been graffitied with an inexpertly sprayed "Fuck Michael S" some time over the past week. It hadn't yet been painted over; small drips bled from the "S" where too much paint had been used. Ryan didn't know who had painted it, but it made him smile to think of how Michael's leadership was sufficiently bad to drive some pallid government employee to break the law.

But to think of someone he liked. Chandra. He had fallen for her, just a little. A small part of him hated that.

She reminded him of Amy, in some ways--reading what Chandra had written, and getting reminded of Amy, had made Amy's absence more intense than it had been for months. A dull, ever-present ache had been transformed into something enormous missing from his life, again. It was like when he did not think he was hungry, and ate a few bites, and suddenly realized her was starving; connecting with her reminded him what a real human connection could be like. An errant phrase could bring back memories long thrust back.

He ejected this train of thought from his mind. Following that line of thought could still... incapacitate him, if he let it go where it wanted to go.

In any event, if this was Hanna's assistant, he had no idea what Hanna must be like. He took out his phone, and shot Chandra an email from his personal address.

*Michael says forget it.*

He clicked send. Two cigarette-pulls later, a response came through.

*And now what will you do?*

*There's no good way through him in the government.*

*And there are no other ways to act?*

Chandra Devarajah's public key was attached for to the next email. A public key for PGP cryptography. With this, he could encrypt messages to her that no one but a government with a supercomputer and a few eons to spare would be able to read, and that was if the government was lucky. He didn't respond with a message immediately, but he did respond with his own public key.

And then took out another cigarette. This was at least a two-cigarette break. And besides, he wasn't sure what else he had to do now that his important work was done; another report on the abysmal security at OIA? He bet a dedicated brute-force attack could have guessed someone like Michael's password in ten seconds flat--Michael was the kind of person who would have used a variation of "Monkey123!" for every account since childhood, even while rigorously enforcing other regulations on other aspects of security.

Chandra was suggesting extra-governmental action, it seemed.

Well, why not? Umbra Labs sounded sinister enough.

He had read a lot more about Alaric, while he was researching Umbra Labs. Alaric had made his first few tens of millions building startups and selling them; he had a talent for letting go when they were at their most hyped and most valuable. Ryan had been able to dig up a number of reviews of him as a leader, from online job search forums. They were uniformly negative.

More than once, Alaric had promised programmers equity or generous compensation packages, conditional on their not quitting the company for a particular period of time. He had then waited until their work was almost complete, and moved the company across the United States. Rather than relocate their families, many programmers quit and lost their compensation.

Another time he had sued to acquire the product a programmer had been making in his spare time away from the office, on the grounds that all the code produced during the programmer's employment was his own, per the Faustian contract his programmers signed. Rather than spend tens of thousands of dollars contesting it, the programmer had given it to him.

Alaric had been married and divorced three times. In each case he had escaped with the entirety of his fortune, due to the prenuptial agreements. Also in each case he had married someone in their early twenties, even as he got older; and also in each case, there were quiet rumors of abuse.

The only thing that made Alaric different from your standard piece-of-shit was that he was intelligent. Intelligent but malicious people always bothered Ryan. He couldn't understand them. Humans liked to be liked; they loved to be loved, or at least Ryan thought that was probably the case. Intelligent people, he thought, would realize this and adjust their behavior accordingly, so that they could enjoy being liked and being loved. When he encountered Alaric, he wondered how horribly, horribly wrong his basic model of human behavior might be.

He snubbed out a cigarette and stepped inside. He continued thinking as he walked back to his pen.

Would he do something illegal, if Chandra asked him to do it? He had no certainty Chandra would ask him. But he could not think of any effective and legal methods of stopping Umbra Labs. As far as illegal methods, though... he could think of quite a few.

He would get fired, if he was caught hacking into Umbra's systems. At a minimum. He would also be prosecuted, and jailed, and so on. Of course, he thought the chances of his getting caught were minimal. But every hacker thought that, including the ones who were caught.

But what did he have to lose? It wasn't as if he loved his job now. What were his current plans? To spend another twenty years doing something he hated? Or to try to return to AI research... when every time he tried to touch their old program, he found himself thinking of a 'we' which was no more and that no effort could bring back.

Better to try something crazy and doomed to failure than try that.

Ryan had heard of people who had tried drinking a soda, and found a dead rat in it. And ever afterwords, they couldn't endure the flavor of soda and had to drink nothing but water; the association had been strong enough to flip around something they enjoyed and turn it into something they hated. He wondered if something like that had happened with AI, when Amy had died.

No, he wasn't going to think about that now.

So anyhow, yeah, he knew what he would say if Chandra asked him to do something illegal.

Depression makes people apt to take stupid risks, the little voice said.

I'm not--

Yes you are.

Oh, fine, I am. You going to do anything about it?

Nothing I can do.

Right. So fuck off.

*The following exchange is encrypted.*
To: ryan.szilard@gmail.com
From: cdevarajah@gmail.com
Ryan,
All the following is my own initiative--not Hanna or Hofvarpnir's in any way. I'll be straightforward.
Are you up to breaking the law to find out what Alaric is up to, and stopping him?
Chandra

To: cdevarajah@gmail.com
From: ryan.szilard@gmail.com
Sure.
What's your plan?

Chapter 4

View Online

"What we can do through our friends, in a sense we can do ourselves."--Aristotle

To: ryan.szilard@gmail.com
From: cdevarajah@gmail.com
Ryan,
Ok. So here's the situation at my end.
I've been trying to get information on Umbra for a while, as you no doubt realized. I've attached a document detailing methods I've used to get into their systems. All of them, however, have foundered on one fact--Alaric keeps the most important parts of his system in a computer physically separated from the rest of the world.
Of course, I could try to get a program into some kind of portable media that will be inserted into that system, rather than through the network. But it looks like he is careful--even paranoid--about what kinds of media he puts into the system. I've managed to compromise a number of the computers and systems in the same building as his code repositories, but so far as I can tell none of these have ever been connected to the main repository, or to the hardware on which he runs the most important programs.
So the problem is one of physical penetration. It's hard for me to do this, given that I live in Germany; and I'd need to take some time for surveillance, etc. But you live just a short drive from the building housing the repository.
So yeah, I need help gaining access to a closed physical system.
I've attached the address and the initial information I have about security in this place.
Chandra

Ryan looked over the building he was planning to break into.

He was taking a long lunch on a Thursday. His boss wouldn't miss him. Or at least his boss wouldn't miss him enough to do more than complain. Government employees were hard to fire. And Umbra Labs was close to work; on his newly-bought motorcycle he could get through traffic a lot faster than he could in the car, especially with occasional lane splitting. And occasional traveling on the shoulder. And occasional creeping ahead along the side of the road at stop lights. The feeling of speed was so much greater in a motorcycle than in a car. The proximity of death or injury helped you feel alive.

He had forgotten how much he liked the freedom of using a motorcycle. Until a week ago, he hadn't used one for... was it four years? When he had been dating Amy, she had been concerned about how dangerous they were, so he had sold his. But he intended to break the law, and living dangerously in one area of life made him want to live dangerously in another, so he had bought a used Kawaski from a seller on Craigslist

He was in a parking lot at the rear of a strip mall, next to his lime-green Ninja. A long grassy bank lead down and away from the parking lot, followed by a short, dusty plain filled with scrub bushes, a completely dry drainage ditch, and kudzu. Then a cleared area with dying brown grass. Then a very tall security fence. A smaller parking lot. And finally Umbra Labs' building.

The parking lot was only three-quarters full, and struck Ryan as small for the building. The building was five stories tall, and apparently Umbra was the only occupant; Chandra said that they had bought it oughtright rather than leasing. Many rooms must be empty, sparsely occupied, or filled entirely by equipment.

Of course, a software company only used a few kinds of equipment. So that probably meant some rooms were filled with racks of servers or GPUs. This was one of the reasons that Chandra had said she was concerned.

Ryan raised a camera with a telephoto lens to his eye. The image of the building that he gained from it was bright, clear, and vibration-free. The vibration-dampening lens and camera had arrived in the mail two days after he had agreed to get into the building, without his even requesting them; Chandra had said that they would be useful when he was scouting out the territory. She had requested that he take extremely numerous photos. "I'm rather a fiend for more data and information about things," she had said.

Ryan found it a little frightening how quickly she worked. Most humans would have moved from deciding to do illegal action, to accustoming themselves to the idea, to planning and working to execute the idea, rather slowly, as their self image shifted from one of law-abiding individual to lawbreaker. Chandra had moved from his agreement, to actively planning with him to break in, in just a few hours.

It was an amazing camera as well.

The fence extended around the entire facility; he snapped a few photos of it. The fence itself was not made of diagonal chain links; it consisted of rectangular panels of dense wire mesh, attached to concrete posts sunk deep into the ground. Video cameras sat atop further poles both behind and before the fence, looking along overlapping sightlines parallel to it. These broadcast to a guardhouse inside the facility 24/7, Ryan knew; it was probable that Alaric was able to access the feed from these cameras at any time. Atop the fence sat an anti-scaling apparatus--they were circular, spiked rings, which Ryan knew would rotate if you tried to grab them while climbing over them.

There was one road in. It lead past a security box for a guard, through a large, sliding gateway and anti-vehicle security pillars rising from the road, to the parking lot. Ryan took a few more photos of the entrance. I need to try to get some from other angles, he thought.

The external security was, frankly, entirely ridiculous. Ryan wondered how much of it for show. Umbra Labs had several government contracts, and he knew that government officials would have been impressed by the security. Or maybe all the visible security was simply a blind, meant to distract any potential intruder with an obvious difficulty so that they would be snared by some hidden security measure.

It was a lot more interesting to break into a building whose security had been designed with real intention.

Moving on from the parking lot to the building itself. The outside of the building was unremarkable: the exterior alternated between ridged cement panels and mirrored windows. There were a few different entrances to the building: it looked like there was a large atrium serving as the main entrance for employees, a few small emergency exits, and a maintenance entrance large enough for a truck to back into it. Each of these appeared to be able to be locked or unlocked remotely; Ryan had never seen anyone use physical keys.

Chandra had told him that the first four floors of the building were used by programmers working for Umbra. There was little additional security in them. Alaric's office, as well as the office for a few other key figures, was on the fifth floor. It took a special electronic key to take the elevator to those levels, she had determined; the emergency escape stairwells did lead up to that floor, but the doors from the stairwells to the fifth floor only opened from the inside. She had also said that the computer programs and source code she was worried about probably only existed on computers in the fifth level, in a room that could only be accessed by passing through Alaric's office. They surmised that only Alaric himself knew the codes required to get from his office into the server room, but all inferences on this point were uncertain.

There were a few sensors in the server room itself, which broadcast information to the outside world: a temperature sensor, a humidity sensor, and things like that. Alaric wanted to take no chances that a fire or disaster of some kind would ruin his computers; Chandra said that she was reasonably sure that there must be GPUs running 24/7 in the room, given these precautions, and given the feedout from the sensors themselves. But though the sensors were connected to the internet, the computers themselves were entirely disconnected from the internet.

God, Ryan thought. The code they sought was just a pattern of information. Probably less than a few megabyte's worth. You could keep it in a microdisk that weighed an ounce. But I apparently live in a world where this is potentially sufficiently powerful and world-shattering that it can serve as the McGuffin in a quest. He liked it.

His phone beeped. That would be an email from Chandra.

*Feeling intimidated by the sight?*

Ryan had granted Chandra permission to view his location information over an app specifically designed to get friends to meet. She had said it would help her provide contextually useful information, and also help her avoid wasting time contacting him when he was driving or on his motorcycle.

Ryan emailed back.

*Never.*

*So you have a plan for how to get in?*

*I'm sure you already have one, Chandra.*

Ryan raised the camera again. A car was exiting the facility. The anti-vehicle pillars sank into the ground as the gate slowly rattled open; the car pulled out, and the pillars again emerged from the ground. He snapped a few more photos.

*Suppose I had no plans. And suppose you were to know you would succeed in getting into the facility. How would you expect that you got in?*

Well, that's a standard psych technique, Ryan thought. Spur creativity by acting as if you've already succeeded. He looked at his watch. He needed to start getting back to work.

*I'll think about it.*

He took a few more photos of the fence and doorways, then zoomed out and took a few more wide-angles of the entire facility and the surrounding land. Then he put the camera in his backpack, hopped on the motorcycle, and left.

That night, he spoke with Chandra over an encrypted VOIP program she had sent him, which she told him was the most secure way there was to talk with him. He ran it on a VM inside his desktop though, rather than on the metal. He thought he trusted Chandra. But he wasn't sure, sometimes.

He had only spoken with her aloud a few times, now, and he was getting used to her voice. It was crisp, with a slight German accent, but no other inflection at all. He had asked about that, and she had told him that her parents had moved to Germany from India when she was very young. Her mother, she said, had been a software engineer, and her father had been a mathematician--and intelligence, she admitted modestly, was to a great extend determined by genetics and early childhood. She had been homeschooled, but had quickly surpassed both parents.

"What exactly is the point of breaking in?" Ryan said.

There was a short pause at the other end. The sound of typing in the background stopped momentarily. Chandra always seemed to be at her computer.

"What do you mean?"

"Do we wish to know whether they have developed AGI? Or do we wish to stop them, if they have?

"Both."

"But suppose Alaric has developed an AGI. Suppose he has created a fully-fledged oracle in that facility--a superhuman AGI--and is merely waiting to connect it to the internet. Alaric would be an idiot not to have backed it up elsewhere, encrypted very hard. And he's not an idiot, so he must have. And that's all he needs to recover it, no matter what happens to the building. As long as he's burned it onto a blank DVD somewhere in the world, and this would mean that there is *nothing* that we can do to stop him, because he can always retrieve it, and just install it on a different cluster of GPUs He will still have an AGI."

"All this is true. But, supposing he has such a backup, what you're presupposing is that he knows that we've broken in."

"Oh. So you want to corrupt--"

"Exactly."

"Wait, I'm not that fast," Ryan said. "You plan to find out how close he is to an AGI, and corrupt his files if he is close to it but has not achieved it. But if he has achieved it, you plan to look into how the AGI is working, and make it appear to be malfunctioning, so that he will not know that he achieved an AGI even if he has?"

"Precisely."

"Let me think about that," Ryan said, and spun in his chair. He bit into a burrito that he had bought from Chipotle. He was trying to increase the protein in his diet, to try to gain a little muscle-mass in case breaking in required some sort of physical prowess. This was probably a stupid idea, but he found he liked it. He also had to admit to himself that he had fallen apart physically over the last few years. He ached a little from motorcycling earlier in the day.

"In either of those cases," Ryan said, "you're going to require some pretty intense and quick analysis of what's going on in his computers."

"Yes."

"Are you sure you're able to do that? I don't know if I could... or I'm pretty sure I couldn't."

"Have you opened the package I shipped you today?"

Chandra seemed to be faster than Amazon Prime, sometimes.

"No."

"Why don't you do that?"

Ryan slit open the brown package she had sent him. A strange... device, slid out, on to his desk; Ryan felt his hair momentarily stand up, because for a moment he thought it was an enormous insect. Then he realized it was a machine.

It the shape of a seven-inch cockroach. It had an articulated plastic spine running from one end of its body to another, the same color as a cockroach shell. Along the side of the spine, six small ducted fans were pressed up against its body; they had hinges, as if they could swing out at a moment's notice. Where you would expect legs or a belly, there were some small nubs that looked like legs, as well as some computer adapters: a USB, an ethernet adapter, and a few other common connectors. There was no visible antenna or on-off buttons on it.

He slid his finger over the case. There were minuscule ridges in it; someone had 3d-printed the device's shell. Even so, all the seams were smooth; it didn't feel fragile. It was an amateur construction, but an the work of a superbly skilled amateur.

"Where did you get this?"

"I contacted a hobbyist in the US and had him make it. So you already know what it is?"

"...no."

"It's a combined hexacopter and universal wireless adapter. If you can get it in the same room as any of the servers, it will be able to plug into them and I'll be able to take a look at them. It has some pretty sophisticated onboard capabilities, in case his computers are in a faraday cage, but hopefully we won't have to rely on them."

"...when did you get this made."

Chandra paused.

"Oh, I'm not sure. When I realized we'd need it," she said.

Ryan had once considered the possibility that Chandra was a bizarre false-flag operation, designed to lure hackers into the most incriminating circumstances possible. He had decided this was impossible, for many reasons: the setup required for this would have been bizarrely intricate; he contacted her, initially; trying to get alarmist transhumanists imprisoned struck him as a program no government was yet likely to commit to. But sometimes her excessive preparation alarmed him; he felt like there was no way she could be merely a single person, given this kind of construction.

Who, really, was he working with? He was glad he had installed her software on a VM rather than directly on his computer.

"A friend of mine designed it, actually" Chandra said, interrupting his meditations. "She's really into black hat stuff, and has used this design a few times before. It's designed to be inconspicuous, as you can see; to be mistaken for what it's not. I sent the plans to a contact in the US, and he was the one who build it."

Oh, well that made her seem a little less omnipotent.

"Are you sure you're up to this?" she said, having for the first time a note of concern in her voice.

"Oh, definitely," Ryan said.

"Huh," Chandra said. "I've worried that you're spending a bit too much time by yourself. People do need regular socialization, to work at their best, and to not become erratic."

"I've already spent a lot of time by myself. I can handle it."

"Were you working effectively, making a lot of progress on your own projects, when you were living by yourself?"

"That's irrelevant," Ryan said.

"So that's a no. You've mentioned that sometimes Braden asks you over for dinner."

Ryan didn't respond for a moment, then spoke.

"What's up with this?" he said. "I don't see how this has anything to do with stopping Umbra."

"If your continued mental well-being is important to stopping Umbra, it does," Chandra returned. She continued: "But I'm not obliged to have no concerns about you, save for stopping Umbra; I'm allowed to worry about you directly. And you've not been well lately?"

"I've been fine."

"I know where you've gone for the last week; nowhere but work and tasks related to our project. Were your weeks before our project any different?"

"...no."

"Thank you for not lying. So how about you accept one of Braden's invitations, then?"

Ryan sighed.

"Alright."

"Great," Chandra said. "Switching back to the regular topic: So have you come up with a plan to get in? Of course I have one. But it is always good to come up with alternate plans, in case you've noticed something I have not."

"Well," Ryan said, trying to forget what they were just talking about, because he was just fine by himself, "There's the problem of crossing the fence without being observed. There's the problem of getting in and to the fifth floor without being observed. And there's the problem of getting into the server room without being observed. Followed by the problem of doing all this in reverse."

"Continue," Chandra said.

"Here's one way we could approach it. The CCTV cameras outside are the kind that use gigahertz-band UHF signals to communicate. We buy an overpowered jammer, or modify a regular jammer, and periodically fuzz the signals from the cameras for a few nights. So they'll begin to be sure that something is wrong with their signals, and it won't make the guards suspicious. After a few nights, when the signal is fuzzed out particularly badly, I get over the fence. I then get inside the building, by inducing the unlock signal in one of the side-door locks; I saw the model, and we could do it. After getting to the fifth floor, I look at the door to the server room, we together figure out whatever kind of security he has installed there, and then I plug this device in in the room."

"Huh," said Chandra, lightly.

"Not very good, is it?"

"It's a little heavy on the personal espionage and flash, light on the certainty that this will all actually work. In particular, I don't like the idea of figuring out how to break into the room in an ad-hoc fashion."

"Yeah. And it doesn't use the little roach that you just sent me. So what's your plan?"

Chandra told him.

Ryan had to agree this was better. His plan had, perhaps unconsciously, been ever-so slightly optimized to give him an opportunity to do some crazy things. This plan was optimized to work.

"Well," Ryan said, "you're right that isn't as sexy. But I'll get started on it."

Chapter 5

View Online

"People who make a difference do not die alone.
Something dies in everyone who was affected by them.
Amos made a great deal of difference, and when he died, life was dimmed and diminished for many of us. There is less intelligence in the world. There is less wit. There are many questions that will never be answered with the same inimitable combination of depth and clarity. There are standards that will not be defended with the same mix of principle and good sense. Life has become poorer."--Kahneman

"Glad you came over," Braden said.

They were sitting in the backyard of Braden's house. Braden was tending the grill with one hand, holding his younger child, whose name and sex Ryan was trying unsuccessfully to remember, with the other arm. The older child was playing a game involving a whiffle bat, a soccer ball, and running back and forth from one end of the yard to the other, often while screaming. Braden's wife, Christine, had just emerged from the house with a plate of vegetables, and was sitting down. Ryan was about 95% sure she was pregnant.

"We haven't seen you in so long," Christine said. "So you aren't enjoying your current job."

"Braden told you that, huh?" Ryan said.

"No," she said, "Three years ago, I think you said you'd rather die than work in job you couldn't code at. Or in which you had to be micromanaged. If I recall correctly."

"Yes... I did," Ryan said.

"So the inference was pretty easy," Christine continued. "Why don't you quit, then?"

Ryan had forgotten how direct Christine could be. He knew that he mentally categorized everyone he met into "can code" and "cannot code," and sometimes the extra attention he paid to the former category made him forget important details about the latter.

"Well, I do need money," he said.

"You could get a job doing machine learning very easily," Christine said. "And you would like it more."

"Yeah..." Ryan began, and paused. "Actually no. No, I wouldn't like it more."

"But you love machine learning, and AI," she said.

"Not really..."

"Ryan," Braden said, "You missed a dozen parties I invited to your in grad school because you were trying to to figure out how to do natural fact extraction from Wikipedia, or make a natural-language interface with reasoning capability for the resulting database, or making sentiment analysis software with language parsing to hook up to a stock-trading algorithm, or--"

"Yeah, I get the point," Ryan said. "I used to. But you've changed what you liked. You used to be nationally ranked in that one online card game, what was it--"

"That's really rather different," Braden said.

Christine smiled.

"Someone else may have persuaded him to change his view on that game," Christine said. "Did someone get to you and persuade you that machine learning was a waste of time?"

Ryan looked down at his hands. "No, no one got to me."

"Huh," Christine said. "So what do you enjoy now, if not machine learning?"

"I'm not sure," Ryan said.

"You see," Christine continued, "I remember after my mother died, my father went through a few different hobbies. He started playing golf seriously. Then he quit that and painted miniature trains. Then he wanted to pursue a pilot's license. He met his current girlfriend during the classes he took for it, fortunately."

"Yeah," Braden said, looking at his hands. "Had he been interested in any of those things before?"

"Yes, he was an amateur golfer before, and always liked machines," Christine said, abstractly looking into the middle distance. "But I gather you're not like that. For you there was only AI research, and Amy. At least once you met her."

"Ah, dear..." Braden said.

"I realize I'm pushing," Christine said, to both of them.

"No, it's ok," Ryan said.

There was a moment of silence. The older child started to sprint back from one side of the yard to the other side, shouting as he went.

"It's hard, when you get your personality so intertwined with another person's like that," Christine said. "You rely on the for so many things that you do, that they feel like another hand. You learn to think inside the back-and-forth of conversation with them. Your self-estimation, your ability to understand yourself, gets tied up with them."

"Dear..." Braden said, "I'm not sure this is helping."

"Yeah, she's right," Ryan said. "I'll take up a hobby. I actually recently bought a motorcycle, and I've been enjoying cruising around in it."

"Really!" Braden said, "You haven't done that in forever."

Braden flipped two hamburgers on the grill, and spoke again: "For at least five or six years, I mean. Weren't you interested in that people-simulation program all the way back then as well? The one that connected to zoning laws, was it?"

"You're thinking of the human interaction modeling system," Christine said. "If you recall, I actually helped him with that; I was interested in computer aided design, then."

"Oh, yes," Braden said.

Ryan momentarily thought over how he had introduced Christine to Braden after Ryan had gone on a first date with Christine, and also thought over how everyone else was probably thinking about that and not mentioning it because it was mildly awkward. Ryan had met Amy just a few weeks later.

"A hobby might help," Christine said, "But that wasn't quite the point."

"So your point was..." Ryan said.

"I don't know if merely a hobby is going to help," Christine said, giving Ryan a look.

"No. I'm not dating again," Ryan returned. "I'm not. I can't."

"Not yet, at least," Christine said, and looked a little bit sad. Then she got up to go to the bathroom, and when she returned the conversation had moved to more pedestrian areas.

That night Ryan dreamed again. This time, he was working on an enormous project with Amy: They were building a railroad bridge, which would carry an enormous passenger train, laden with all Ryan's friends, from one end of a ravine to another. The ravine was deep, with a narrow thread of water barely visible behind massive, jagged rocks jumbled together in the gorge. He could see the train coming towards them quickly, in the distance. But he was working smoothly with Amy. Their movements were coordinated and efficient; it was hard work, but by the smile on her face he knew they would finish it in time.

And then, he looked up from hammering and he couldn't see her any longer. He looked on either side of the bridge, to see if she was clinging to the edge, but she was no longer there. There had been no sound. He felt panic grip him, like a giant hand had squeezed his heart. She had just been there. She had been there seconds ago. Had he messed up? It must be his fault that she was gone.

The train was bearing towards him, but for him that just added a little to his misery; it was just the manifestation of that enormous mistake he had made, somehow, without knowing that he was making it. By the time he came to know that he had done something, the mistake was already complete; he wanted to scramble at the time now irrevocably gone, but he could not.

A few weeks later, Ryan interviewed for a job at Umbra Labs.

Ryan had polished his resume, and puffed it with the unfinished side-projects that he had listlessly pursued over the last two years. Chandra had helped a little with that; things would look a little more finished than they originally had been. He spoke with a tech recruiter from Umbra, performed a moronic programming task online, and finally got to the on-site interview. At one point during the set-up for the interview, he specifically asked if he could meet with the head of Human Resources to talk about some of his concerns about the working conditions at Umbra Labs. At this point, he had caught the attention of those hiring him--his academic credentials were excellent, even if his history of shipping products was questionable--so of course they said yes.

The interview itself was extremely uneventful.

Ryan drove his car there. Security let him in, checked him into the building, and accompanied him everywhere he went. He journeyed to the second floor, and programmed an in-place quick-sort algorithm in C++, as well as A* in Java. He answered numerous questions about Sansa reinforcement learning, and sketched out the architecture for a system meant to use unsupervised learning to enhance a supervised learning task. His interviewers were uniformly impressed.

Afterwards, he visited Human Resources on the fifth floor and expressed very mild concerns about the CEO. HR assured him that everyone was fine, that the stories were from disgruntled employees, and that Mr. Comtois was a wonderful individual whom she personally knew very well and who would never behave anything like the rumors said. All the rumors were from a few disgruntled employees who had made it their mission to smear him, she said. Ryan smiled when he was talking to her and wanted to spit when he left.

A guard had accompanied him the entire way. The guard was humorless, omnipresent, and silent. In the fifth floor, though, Ryan stopped momentarily to use the bathroom, which the guard was willing to stand outside of. While he was in the stall, something hummed and skimmed out of his pants leg; it sucked itself on to the wall, and skittered roach-like upwards. On the ceiling it pried loose a panel, slid through, and let the ceiling panel down again.

Then he left the building, checked out with security, and left the grounds. He had seen the outside of the door to Alaric Comtois' office, and nothing more.

That night, he asked Chandra if the bug had gotten into the server room.

"I am executing the contingency plan we made, in case the bug was initially unable to get into the server room."

"Ah, I figured it was too easy."

The ceiling of the server room was closed off: the roach-device couldn't find a ceiling tile to pry up in order to get in.

So the plan called for Chandra to alter the readings that Alaric was receiving from the temperature sensor in the server room. After all, for these readings to be any use to him, Alaric had to be able to receive them anywhere. So this system was connected to the internet, even though the computers in the room were not. And what was connected to the internet, could be altered by it--or at least spoofed by it. Somehow Chandra had managed to gain control over the stream of data Alaric thought he was receiving from the sensors.

And then she increased the temperatures he received slightly, as if something were going wrong in the room. Alaric would be able to see the increase, and he would worry that a fire had started or something had overheated. Alaric, however, was currently on the West Coast. He had no choice but to call security, give them whatever information was necessary to get into the room, and ask them to check it out. At which point the roach, which had by then secreted itself in Alaric's office, would slip in, and do its thing. That was the plan.

Ryan wished that he could have seen Comtois' face. The need for everything in the room to remain safe, balanced against the displeasure he would have at letting anyone else in; it would be torture. It's very hard to keep something safe, though, without trusting someone else.

"Ah, he's called security," Chandra said. "They're heading upstairs."

"Why are you doing this, Chandra?" Ryan asked, as if the question had just occurred to him.

"I thought we had gone over that," Chandra said, sounding a bit distracted. Ryan could picture her thin... pretty face, looking over a computer displaying every bit of information she could suck out of the multitudes of systems at her control.

He didn't trust that image. He felt, somehow, like he had been given that image. They typing of her keyboard, the occasional background noises he heard--something felt weird.

"Humor me," he said.

"Well, I like to stay alive. And I like other people to stay alive," she said, still sounding quite distracted.

"Of course," Ryan said. "But over the past few days, you must have put an incredible amount of effort into this. Does Hanna just let you take time off, so you can modify blueprints for roaches and hack into climate-information systems and so on?"

Saying it out-loud made him wonder about it more.

"What was that?" Chandra said, "Sorry, I'm still watching what's going on, here. It's a little involving."

Hanna, surely, must have noticed this activity. No? Or maybe not. Chandra might be the kind of person who simply didn't require any sleep, or who survived entirely on espressos and Mountain Dew and Modafinil. That, perhaps, would account for the kind of output he had seen from her. Perhaps.

"Oh, never mind," Ryan said.

What are the possible hypotheses about this?

One possibility: Chandra was a brilliant, altruistic hacker who independently, and apart from Hanna or Hofvarpnir, just wanted to stop dangerous AIs from coming into being. Evidence for: This is what Chandra says. Evidence against: Brilliant, polymathic hackers are more common in books and movies than in reality.

Another possibility: Chandra is actually working for Hanna and Hofvarpnir. They are going to ship an AGI in the Equestria Online MMO. They went through all this effort because they want that AI to succeed in whatever tasks they've set for it, and need to guard against a rival AI which could compete with it. Evidence for: Makes sense of the resources at Chandra's disposal. Evidence against: Ryan knew, from reading Hanna's own articles and writings, that she was acutely aware of the dangers of AI. They hadn't shipped an AI earlier, when they had created Loki. Why would they do it now?

Another possibility... there were too many possibilities.

You need to wait, Ryan told himself. You just don't have enough evidence to nail anything down yet. What is clear is that you've been moving too fast.

Time to get some information on Chandra and Hofvarpnir.

To: alexander.yao@stanford.cs.edu
From: ryan.szilard@gmail.com
Hey, any news on those papers I sent you?

From: cdevarajah@gmail.com
To: ryan.szilard@gmail.com
Ryan,
Great news. The roach was able to sneak in after Comtois sent some security guards to check on the room; he had to tell them the security codes necessary to get in. Today he plans to change all the codes, but by now it doesn't matter. The roach got in, plugged itself in, and slid beneath a server case. Unless he moves everything in the room he won't find it.
He was very close to AGI, as far as I can tell. He's been using the the power the rooms affords to sort through different conceivable AGI architectures dynamically, and evolve them on their performance. I've begun to tweak his code; he won't get any further.
So congratulations! We've done well. Now we just need to search for the next possible AGI threat. I've attached a document with a few different companies for you to look into. I think you probably should pay particular attention to Alfheimr. They look well intentioned, but I'm not sure about how responsible they are; they have a kind of lightness in pursuing something of this gravity that I don't trust. I'll send you more information on them when I get around to it.
Chandra

To: ryan.szilard@gmail.com
From: alexander.yao@stanford.cs.edu
Ryan,
So sorry this took me so long. Anyhow, about the article you sent me.
I'm not sure what to make of them. As far as I can tell, it is impossible to implement the algorithms outlined in these papers. The papers basically consist of non-constructive proofs that particular mathematical methods are possible, but they don't explain how to actually implement these techniques. To implement them you'd need to make breakthroughs in several different areas of computational analysis.
So they aren't really relevant for the kind of thing you're interested in, unless you have a pocket genius who is able to implement them. Of course, if you can, then you'll be able to do all sorts of fun things with them. But that's true of a lot of results in math.
I can go into this further in detail, if you want. I've attached annotated copies of each of the papers with further thoughts on each.
Thanks,
Alex

Ryan read Chandra's email at work. Huh. He momentarily looked into Alfheimr--but they seemed small, idealistic, and entirely out of their depth. His gut told them that they could not get anything like AGI. So he decided to look at Hofvarpnir instead.

Ryan wasn't sure what to look for, but he thought Hofvarpnir's finances would be a good start.

Things began to leap out at him.

WorldFoundries had recently announced that they had partnered with Hofvarpnir to make the dedicated microchips for the Equestria MMO. Apparently the Equestria MMO would run on a dedicated device. WorldFoundries was a relatively small integrated circuit foundry; they made microchips for some computers, but mostly for cell phones. Small was always a relative term when applied to foundries, though; a fabrication plant for even a medium-sized foundry could easily cost upwards of a billion dollars.

Their announcement of this partnership had not been terribly public, in any event, and had been noticed by no one outside of the foundry industry.

But Ryan also found that WorldFoundries was breaking ground on a new chip fabrication facility. This would be located in Germany, like WorldFoundries' other plant.

When integrated circuit foundries built new plants, they generally announced why the plant would be awesome--how they were shrinking the die size, and so on and so forth. It was often a big affair, with a lot of publicity to attract new customers. The WorldFoundries website didn't even announce that they were building a new plant, however. Ryan had to dig up the information on a corporate research website.

And they were proceeding quickly. The sub-basement would soon be completed.

This pace would take a large amount of money. How could WorldFoundries afford this?

Ryan began to search through contracts, documents of sale, agreements, and miscellaneous financial data. Several times, he had to pay -- or at least have the government pay -- so he could use some of the curated, expensive services which collected such data. A few other times, he had to use internal government databases and records to find data which no one sold. There was a reason his job required a security clearance, TS-SCI.

In any event, it appeared that WorldFoundries was not, strictly speaking, going to have complete control over the new facility. In fact, it looked like Hofvarpnir would have exclusive use of it; apparently they just needed WorldFoundries' expertise in setting up some of parts of an IC fab. For the next eight years, they would be the only people who used it. At which point control of it would revert to WorldFoundries.

This was bizarre for many reasons, not least of which was that Ryan had no idea how Hofvarpnir had enough money to do this. He returned to Hofvarpnir's finances.

And so he began to search through the same expensive services, and internal government databases, and records, to see if he could determine how much money Hofvarpnir had at its disposal. And, after a few days, he began to come to a somewhat alarming conclusion--he had no idea, and no one had any idea, how much money Hofvarpnir had. Hofvarnpir was privately owned by Hanna, which gave it a great deal of discretion in how much information to publish. And it was currently exercising that discretion.

He kept looking to try to figure out what other entities Hofvarpnir was dealing with, and whether it had diversified its holdings into other areas. And once more, very quickly found himself out of his depth. It looked like Hofvarpnir had suddenly, and very recently, begun to enter into numerous Byzantine contractual arrangements with a number of actors. Robots manufacturers. PR firms. Web design companies. ISPs. Several university-centered research facilities. Each of these arrangements he could only dig up after enormous effort, going through occasional shell companies and mediating entities and third-party controllers and offshore accounts. After all this effort, he could only make educated guesses at most things. Other things he found were just hints. It took him days to track down each thing, and there were many things to track down.

But, in any event, it looked like, about the time he had started to talk to Chandra, Hofvarpnir had begun to expand its reach immensely. More than any mere video game company was likely to do, unless it planned to be something more than a video game company.

He found that he was angry at Chandra, as well.

He told her that he was looking into Alfheimer. He also told her that work had picked up, and he didn't have as much time to do things. She seemed to take this mostly in stride. None of her emails seemed suspicious or wary. She said that she had been most worried about Umbra Labs, and that her concern with Alfheimer was really out of a desire to be absolutely punctilious in her work. She also said that she was happy to volunteer information to Ryan about her progress on shipping the Equestria MMO. What she sent Ryan seemed accurate, but Ryan no longer trusted the appearance of accuracy; he now believed that she could have sent him reams of documents, all of them absolutely true, while hiding tremendously important things about Hofvarpnir.

His anger was sufficient that he stayed up till two or three many days, working on the report. He hacked into various corporate systems to try to find if they were working with Hofvarpnir, and meticulously figured out how to work the information that he got from illegal activity into the report in a clean fashion. Twice on the way to work, he nearly got into an accident because he was motorcycling too recklessly. His diet switched back from semi-healthy foods into entirely unhealthy foods, as coffee and Monster drinks began to dominate his fluid intake.

Ryan knew he was sliding, but he wasn't that worried about it. After all, it was important to finish his report. After it was done, he could return to a healthier state of being. But for now, his ability to focus and to finish was what was most important.

Chapter 6

View Online

"We're even wrong about which mistakes we're making."--Carl Winfeld.

Ryan leaned back from his computer. His eyes watered. His mouth felt like it was filled with cotton. His brain felt like someone had put it through a blender. His joints ached when he moved them. But the report was finished.

There was a real chance Michael would push this upwards, and that the government would move against Hofvarpnir, in Ryan's estimation. A real chance. Probably a chance in the mid double-digits. A 20% to 70% kind of a chance, which was about as good as he could reasonably expect.

Obviously, it had taken him a long time to compile the report, especially given the difficulty of the material. He didn't have Chandra's speed. He hadn't had Chandra's help. But this report was thorough.

It said the kinds of things that Michael would want a report to say. It cited documents produced by Michael, which Michael would like. And Michael wouldn't have any scruples moving against Hofvarpnir--they were the kind of organization he disliked, the kind of organization that had never contracted with the government for anything, the kind of organization that had been started by a single genius and grown exponentially with time. Ryan had included statements that Michael would read as implicit praise of himself. It was a masterpiece of flattery, as well as a masterpiece of research. Ryan had never brownnosed so much in a single document in the entirety of his time in the government, but he had pulled out all the stops this time.

Ryan Szilard, slayer of AIs. That sounded good.

He read through the report one more time. He wanted to be careful; he had performed many complete read-throughs before. His field of vision grew fuzzy every now and then, and he realized that he was unlikely to catch anything he had not caught before. But that would probably be fine anyhow, right? So he sent it off in an email to Michael, being sure to include careful flattery of Michael in the email as well. Then he left for a smoke break.

By the time he returned to his desk, he had a new email from Michael. "Come to my office."

Huh, that was odd.

He walked, a bit more slowly than usual, down the rows of cubicles to Michael's office. It was only 11:30 in the morning, and he had gotten about four hours of sleep the previous night. He swayed gently as he walked, occasionally putting a hand on one of the cubicles he passed by.

Unscheduled meetings with Michael were quite rare. Michael may have been an asinine boss. But the upside of his particular brand of asininity was that he was only very rarely spontaneous. Ryan didn't see how Michael would have been able to read the report already, which meant that he didn't see any reason for Michael to be asking to see him. Which made the entire request for him to come to Michael even more odd.

Michael's door was open, so Ryan walked in. Michael looked up from his computer, and Ryan immediately realized something was wrong. Michael looked angry, even furious. But Michael also looked happy, which was maybe more alarming.

"You," Michael said, "are going to be fired. I'm putting in the paperwork now."

"What." Ryan said.

"Clear out your cube now. I want you out of the building before noon. You'll two more weeks of pay before your termination is finalized, but you're never to come inside this building again."

"What?" Ryan said. "I don't understand."

Michael looked a little frustrated. His level of anger was clearly calibrated to be met with an equivalent level of anger. He looked like he had been looking forward to screaming at Ryan and being screamed at in return; he looked like he had been looking forward to all the emotional release such an occurrence would afford. Ryan's emotionally dulled state seemed a little frustrating to him.

"What do you mean, for what?" he snapped. "You idiot. For this, for a start."

And he spun the computer around.

Ryan saw the email he had just sent to Michael. But it looked different. He caught some phrases in the body of the email. Phrases like "incompetent fuck" and "mouth-breathing, foot-dragging little shit" and "completely and hopelessly incompetent leadership," all said of Michael, all apparently said by Ryan himself, swam before him.

He knew he was tired... but he was fairly sure that he had not just said those things to Michael.

"Yeah, the email you just sent," Michael said. "The one where you insult me, my mother, and the entire government."

Ryan's vision felt like it was closing around him.

"I didn't send that," he said. Polite discourse abandoned him, and he asked Michael, "Did you fake that?"

"No, you idiot!" Michael said. "And you CC'd ten other managers here, so don't even pretend that I was able to do it."

"That does seem unlikely," Ryan said. Ok, so Michael had not faked this; Michael's technical skills were not good enough to do that, certainly. Or almost certainly. Who knew. But if Michael was too stupid to do this, where had this email come from? Ryan wanted to think, but his mind felt slow; it was like he was trying to run through twenty inches of water. Perhaps he should not have pushed himself so hard recently... but who could have done this?

"And that's not all of it," Michael said. "You were an incompetent criminal before now.... are you even listening to me?"

Ryan wasn't. He was trying to think of someone, anyone, who could have done this and who had the motivation to do this. He had no enemies at the office, that he could think of. He didn't even know anyone at the office, let alone know anyone well enough to make them an enemy. And this would have require at least a little technical skill--more than a little. And who outside of the office, who possessed technical skill, would have had any desire to ruin his career, or to render the work which he was creating--

Oh. Right.

Ryan laughed to himself, loudly and suddenly and unsteadily.

"What!" Michael screamed. This too seemed funny to Ryan.

"I'm sorry," Ryan said. "My bad, I shouldn't have interrupted. What else did I do?"

"I have the security footage of you spray painting the outside of the building!" Michael said. "The place I sent it to isn't done with its analysis. But it looks like you, and I'm sure they're going to find that it was you."

"Of course," Ryan said. "Of course the cameras would show that I was the anonymous vandal. Is there anything else I did?"

"Do you have anything else to confess?"

"Ah, no, sorry, my bad," Ryan said. "I'll get my stuff."

And Ryan spun on his heel, heading out of his office, swaying a little bit as he went.

It didn't take him long to gather his materials. Apart from his report and materials he had gathered while working on it, there was nothing on his desk; certainly no photos or memorabilia. And then he remembered that the report itself was now proprietary to the government, and he had to leave it behind. So he left with empty hands, more or less, even before security came to escort him outside.

His superficial hilarity began to disappear while he drove to his place.

Chandra. She must have done this. He didn't know how she had broken into their email system--there would probably have been dozens of ways--but he was certain that she was the one who had done it.

Given that she had done it, it must have been because she didn't want to run the risk of Ryan's report shifting the power of the United States government against Hofvarpnir Studios. This meant they had something to hide. And this meant that the Equestria MMO would contain an AI. Chandra would need to tar the very idea of investigating Hofvarpnir with bad associations, so she had made sure that no report by him would ever be taken seriously.

He was angry at Chandra. Sure, he had liked her a little--but she had seemed kind, and interested in him, and her incredible talents couldn't help but turn him on a little--

--her incredible talents. Wait.

Her incredible talents. She had seemed superhuman to him, but somehow he hadn't put two and two together. So she wasn't a human, because he already knew that Hofvarpnir had developed an artificial intelligence, which meant that the balance of probability was that any entity with that level of intelligence was not a human at all. God, that was so obvious, how couldn't he have seen it? If I can do so now, with my mind gummed up, how could I not have figured out earlier?

So the superintelligence had been around since at least as long as he was talking to Chandra.

That was quite a while.

Ok. It would have taken, a week or two for it to buy more processors. Assuming it couldn't offload too much of its calculation to AWS. So it would have had banks of a few hundred NVIDIA GPUs to think on, extremely quickly. But given the acquisition of WorldFoundries--who knows--maybe it already had custom hardware to run on. Custom hardware would make it smarter, and put it yet further beyond his ability to understand it and plan against it.

But really he could be projecting this all wrong. It might be that the most efficient way for it to gain processing power quickly would have been to send a few Hofvarpnir employees to a warehouse somewhere, have them pick up every processor, then hire a few highschoolers to plug everything together. Or maybe to make a botnet to distribute its calculations over ten million computers around the world. Or maybe. Or maybe. Or maybe. He realized that his thoughts were going everywhere and nowhere at once.

I can't outthink it, at least like this, Ryan thought.

He stopped at a park that was not too far from his apartment. He walked with his phone to the center of the park, and hid it in the crook of a tree, leaving it on. Let Chandra--it--whatever it was--think that he was thinking over his life while looking at nature.

When he arrived at his apartment, his mind was still spinning. All the things Chandra had ever suggested, when they were discussing the danger of AI, rushed through his head. But now he did not know how many of them to trust. None of them? But if none of them would work, then he would be able to rule out possibilities because she had said earlier, and she would not permit that gain in knowledge. Perhaps all of them would work? Surely not for the same reason. So some of them--but he had no way of knowing which. If the AI was a true AI, it could have picked a random number from the air and used that to determine how many and which methods to describe to Ryan, given a particular ratio of good methods to bad methods. Humans sucked at random number generation; a computer could choose some method of pseudo-random number generation much better than a human.

He needed to calm down. What did Chandra--he gave up, and decided to call her Chandra, for now--expect him to do next?

Presumably, try to stop her.

He tried to think. He sat in front of his computer, and researched for a few seconds. And then stopped, then restarted again. His mind could not stay still. He considered the probability that she had bugged his computer. He considered the probability that he would be able to find a computer to work on where she could not track him. He considered the probability he could get useful resources in a way she could not track, or rally friends without her listening, or accomplish anything without her smearing him or doing something to preclude any further progress.

He couldn't think of a thing.

He started to compile the things he did not know.

He did not know what values Hanna / Chandra / it would possess. He did not know how long it had been in existence. He did not know how many computer systems it had penetrated. He did not know how many resources it had at its disposal. He did not know how many inroads it had made into politics and into influencing powerful humans. He did not know what number of genius-level human intelligences it was now equivalent too.

He turned to things that were probably known.

He did know that, if he had been in its situation, by now he could have rendered himself almost invincible. So it was almost invincible. He did know that it could convince humans to do what it wanted with superhuman acuity. He did know that it had millions and probably billions of dollars worth of assets at its disposal. He did know that it was by now probably running on hardware customized for its performance. He did know it would have secretly spread itself across the entire globe. And he did know that it had just decreased the probability that an organization like the United States Government would investigate it, by a significant degree. Which meant that he did know that it was alert, watching for even the smallest of threats, and was ready to neutralize them.

He had known this feeling of helplessness before. The feeling that everything had been fine, at some point in the past. And then he had screwed up, without knowing that he was screwing up, before his mind was even remotely aware that it was possible to screw up in such a big and permanent way. Like finding that a power tool had removed your hand, before you realized that you had turned it on. Like driving in the rain--

This time he could not get it out of his head. It was still sunny outside, but all he could think about was finding himself in the tangled wreckage of a car, rain beginning to fall through the broken glass. The weirdly delayed sensation of realizing that he had just been in an accident. The sort of abstract debate, in his head, of whether the accident had been his fault. Then, looking to his side, to the passenger's seat, to see--

There was one way to stop himself from thinking about this. His mind flit to the pistol he owned. It wasn't like anyone would miss him, anyhow.

He got up, and began to feel beneath his bed. The gun case was next to his shoes, and he took it out. The lock was still set to 0-0-0. He had not bothered to change it from this default when he bought the gun and case six months ago, when he was still able to tell himself that he was interested in target shooting and somehow to studiously ignore his other motives.

I want never to think about myself again, Ryan thought. Never to have to go back over the unalterable film-strip that was his life, to review the decisions that brought me here. Never to think that the one who made those mistakes in the past was the same as the one who here existed, and breathed, and ate, and could do nothing about the past. To remove the swollen, tumorous thing that was his memory, and replace it with blankness. He craved non-existence like he had learned to crave a cigarette or alcohol.

Somebody knocked on his door.

"Ryan," Braden's voice said.

Ryan sat on his bed with the case in his lap.

"Ryan!" Braden's voice said again. "You aren't answering your phone."

Ryan sat on his bed with the gun.

"I saw your car in the lot," Braden said. "I know you're in there."

Ryan sat.

"Unlock the fucking door, Ryan," Braden said.

Ryan shoved the case beneath the bed again.

"If you don't unlock the door, I swear I'm going to knock it down," Braden said.

Ryan opened the door.

Braden was still wearing the suit and tie from work. He was breathing hard, as if he had run here from his car.

"I heard that you sent some kind of crazy email, and had been filmed graffitiing the building," Braden said, coming into the room.

"Yeah, that's what they said," agreed Ryan.

"And then another friend of yours called," Braden continued. "I hadn't heard you mention her before. She had an Indian name, I forget what it was--"

"Chandra."

"Yeah, Chandra. And she said you were depressed... " Braden said, and he voice trailed off.

He and Ryan looked at each other for a moment.

"She said you might kill yourself," Braden said. "She was really blunt about it. She actually said that you would probably kill yourself, if I did nothing. And then you wouldn't answer your phone, so I came over here, which is where she said you probably were."

Ryan said nothing.

"Are you about to kill yourself?"

"I don't... I'm not sure," Ryan said. He wondered why Chandra had told Braden that he might kill himself. That would be... weird, for her to do that.

"I see," Braden had said.

Ryan sat down on his bed, and Braden sat down next to him.

Braden sitting there felt stabilizing. His thoughts raced a little less. And one came to the forefront which had nothing and everything to do with the last few hours. Ryan had never told anyone, but he needed to tell someone.

"I killed Amy," Ryan said. "No, listen. I was driving. The road was wet. I know that other car slid out, slid into us, but I know that I heard it before it hit us. I had at least half a second, from the time I knew something was wrong to the time it hit us. Maybe even three-quarters of a second, or a whole second, or even more than a whole second. Human reflex time is less than a fifth of a second, right? It's significantly less than a fifth for trained actions. I could have done something, anything. I can prevent a motorcycle from sliding out with less warning, so I could have done something."

Ryan was crying freely now.

"But I didn't. I panicked. I yanked on the steering wheel, and we hydroplaned before other car even hit us. If you look at the pictures, it barely tapped us--it was me who threw us off the road. And then when I realized we had stopped, we had hit the support pillar of an overpass. I was fine, and she was dying. I know they found that other driver at fault, but I could have saved her. So I basically killed her."

And Ryan cried. It wasn't embarrassing to have Braden there, oddly. The misery he was feeling at having fucked up everything filled him up completely, and didn't leave room for any lesser sadness.

Braden put his hand on his shoulder.

"It wasn't"--Braden started, but Ryan interrupted.

"As if you'd ever say anything different," Ryan snapped.

He knew how these things worked. People said it wasn't your fault. But it didn't matter what they would say, unless they had the kind of honesty necessary to tell you that it was your fault, if it were. But he didn't think he knew anyone with that kind of honesty.

He had killed everyone he knew with that kind of honesty.

"It really isn't," Braden said.

Ryan still cried, because even now--as he saw Braden eye the gun case that he had failed to slide completely beneath the bed, and as Braden took out his cell phone to call an ambulance--he knew he was alone. And Ryan thought that---after Braden had gotten Ryan involuntarily committed for a few days; after Ryan had learned to take pills to stop himself from feeling so empty; after he had learned to stop expecting things from life--after all this, Ryan thought there would be nothing that he could find which would justify his continued existence, though his existence would still continue.

Even with Braden here to pull himself back from any self-destruction, the future was empty, after all. He couldn't save someone who was dead. He couldn't stop the AI. There was nothing to do, and for people like Braden he was nothing but a burden, whose continued existence Braden would work to guarantee but for whom Ryan could do no real favors. Ryan could see himself continuing to be, through inertia and the efforts of others and through medication. But there could never be any reason for him to continue to live.

Chapter 7

View Online

PART II

"Have I any pleasure at all that the wicked should die? saith the Lord GOD: and not that he should return from his ways, and live?"--Ezekiel 18:23, KJV

Ryan got up at exactly 6:30, as he had for the last two months.

The first thing he did was take an antidepressant. The second thing he did was add some logs to the cast-iron stove sitting in the cabin. It was cold work putting them in, but the cabin would be colder if he didn't do it right away. Upstate New York could be unforgiving in the winter; people had died of cold after getting lost not far from where he lived.

The third thing he did was get ready for his morning's cross-country skiing run. This too was as much a matter of necessity as much as exercise, just like taking the SSRI and tending the stove. Endorphins were a requirement, not a luxury, after all. He put on a breath-warmer as he wrapped up to go outside; it was about ten degrees outside, and otherwise his lungs would burn from the air.

They burned anyhow, as he slid through the pines loaded with snow, past boulders, fallen and half-fallen trees, along the same path he had taken for nearly three months. His lungs hurt, but the hurt was good. It kept you awake and made you feel alive, just like the cold did. You needed some discomfort to keep yourself awake, it seemed; a little measure of unhappiness to stave off the greater and more overwhelming unhappiness that might otherwise engulf him.

Thoughts attacked him while he skied, but he had learned not to fight them. You're a horrible person. Keep skiing. This lifestyle is unsustainable. Keep skiing. She died. Keep skiing. Everyone dies. Keep skiing. Eventually he managed to get into a rhythm, and all thought disappeared as he slipped into a mindless mental place where all he did was keep moving. Keep skiing. Keep skiing. It was almost as if you were flying when you got on good snow, in a good groove, where the flow of the activity made everything else in the world disappear.

Far from the cabin, he reached a summit at the top of the small mountain he was staying on. The visibility was excellent. He stayed for a while, looking at the scenery. The sunrise colored the sky red and orange and pink; the evergreens around him were beginning to brighten and to look alive. The greenery and isolation of it reminded him of where he had grown up, before he had devoted his life to artificial intelligence and computers. He came to the summit every morning, and every morning had found that it still gave him energy. One week when he had been sick had been the worst; staving off depression without the outside and without exercise had been almost impossible.

It was nearly eight when he got back; it was by then bright and blindingly white from the sun and the sky and the snow. The stove was now quite hot; he boiled water, and made oatmeal mixed with dried figs. He sent an email to Braden, to assure him that he was alive. He sent another one to his shrink, to assure him of the same. Neither of them had been particularly receptive to Ryan's idea of isolation, but Ryan had managed to persuade them that time away from people could recharge him rather than drain him. Braden, in particular, had worried that Ryan would do badly without friends.

After breakfast, Ryan turned to the laptop resting on a small desk. He had work to do. It was web-design work--he had picked it up the programming necessary to do it well relatively quickly. Braden's family was letting him borrow the cabin, and Braden had said that he was under no obligation at all--but Ryan felt obliged to pay rent, and web-design was profitable as long as you had a working internet connection and a steady source of clients.

Moreover, the kind of work he was doing was relatively mindless. He started working a bit before nine, listening to an ambient music playlist. Time disappeared as he worked on forms, payment systems, login systems--and before he knew it, it was one. That was the nice part of this kind of work. So many things were standard in web-design that he could just sit back and let himself work without thinking about it. Just refactor, refresh, and refactor, over and over again.

He ate lunch at one, and added another log to the fire. Months worth of firewood was stacked behind the cabin; Braden's brother, or Braden himself, had been very diligent splitting logs earlier this summer. Ryan had added to the pile himself; splitting wood was another physically demanding task, one which required a kind of flow if it was to be done well. Apparently Braden's extended family liked to vacation here in the summer--there was a lake nearby, which would be very pleasant during summer months. But there was little reason for any of them to visit in the winter, and so it did not inconvenience them to let Ryan stay there.

Ryan had not known what to do with himself when he got out of involuntary confinement, initially. For a few weeks after getting fired, he had stayed at Braden's house, sleeping twelve hours a day. Playing with Braden's children had helped a little, but he found that he wanted to get out of the city and away from all other humans. After cautiously exploring Ryan's motivations for several more week, Braden had agreed to it and helped set him up in the cabin.

At three in the afternoon, he stopped working. It was Wednesday, and on Wednesdays he got the mail.

It took much less than an hour to ski the five miles to the base of the mountain, and pick up a few letters and a mostly-flat box from the mail. But it was about five when Ryan managed to arrive back in the cabin. He started dinner--just eggs and sausage, given that produce was a luxury. After dinner was finished, he sat down, shoveling the eggs into his mouth before they got cold. Only after the food was done that he opened the package.

There was a box inside, colorfully decorated with ponies, unicorns, and pegasi. "Princess Twilight Ponypad," it said. And on the box there was a yellow post-it with Braden's handwriting.

Ryan,
I know you haven't played many games recently, but this one is all the rage now. The game reviewers cannot stop talking about it; my youngest loves it, and I'm going to confess that I've given it a shot. It occurred to me that you might like it as well, so I bought you a copy.
Braden

And you think it would be good for me as well, I'm sure, Ryan thought.

He looked at the packaging, which had the same kind of advertising copy on it as the announcement made by Hasbro so many months before. "Friendship-based gameplay." "Infinite procedurally generated worlds." "Natural conversation with non-player characters."

He looked at one of the drawers on the desk. Before he had left for the cabin, he had snagged a flash drive which contained the entire repository that he and Amy had been working on: it contained all of their code for artificial intelligence, all of their notes about the code, and all of the relevant mathematical and theoretical papers. He wouldn't have trusted the repository which resided on his old computer, at home, because he had downloaded materials from Chandra onto that computer. But he had made this copy before Chandra had ever talked to him. This one was surely intact.

He hadn't worked on it, though, since he had moved here. Sometimes, when he was angry, he wanted to email it to everyone he knew. To go back into the world, and say what he knew of AGI, and try to start the seed of something which could work against the monster Hofvarpnir had brought into the world. Several times, he had been nearly sufficiently furious to try some such desperate plan. But he always stopped long before execution. He suspected that any monster he summoned up would lose to Chandra; and he knew he had not solved the value-alignment problem himself.

He looked back at the boxed device--the PonyPad, apparently. Braden hadn't known about Chandra and Hofvarpnir and the impending end of the world. Ryan hadn't told him; he hadn't seen any point. The well was poisoned; the AI had done what it wanted to be done, and Ryan did not know how to stop it; he saw no reason to ruin Braden's career as well by enlisting him in the same cause. So Braden had innocently decided to send him this, just to cheer him up, presumably.

Ryan wouldn't deny that he wanted some cheering up. Living in nature had, in many respects, been wonderful. It was letting him remember a part of himself that he had long forgotten. But living without regular conversation was trying.

Well, why not? He might as well see what this world-conquering game was like.

He turned on the pad, connected it to the internet, and set it up on the stand. The gamepad controls seemed basic enough. He turned it on, entered a username and password, and entered what he supposed would be the character creation screen.

It didn't start by asking him what statistics he wanted--it began by asking him a series of multiple-choice questions, nearly all of which seemed extremely pointless. "What is your favorite flavor of ice cream?" "When did you last look at a sunset for more than a minute?" And the bizarre "When was the last time you had sex?" Braden had thought the game was for small children; but perhaps the questions were generated uniquely for everyone.

He came to the race selection screen: earthpony, pegasus, or unicorn. There were small descriptions of each, beneath abstracted colorless wireframe models of each race. He chose pegasus almost without thinking: unicorns were able to work magic, true, but magic apparently took a great deal of conscious effort, deliberation, and considered weighing of hypotheses. Pegasi could fly.

And he found himself dropped off in the middle of the mountains somewhere.

His first thought was not that the graphics were beautiful. His mind leapt right over that thought, and instead to the idea that he was in a beautiful place. It was like he had a window into the Swiss Alps, or better yet, into one of the composited mountain landscapes that 18th-century painters liked to form from the alps. He could have sworn he had seen similar towering vistas in art galleries.

He was far, far above the tree line. Beneath him, a blue river meandered wildly between the green foothills of the mountains. The land had been cultivated; a patchwork of narrow fields dominated the flat bottom of the valley. Away from the meanders of the river and the cultivated valley-bottom, forest sprouted up the sides of the mountain. And further up, yet still beneath him, at the border between the tree line and the white slopes of the mountain, a tiny village had been perched on an isolated column of rock, more suitable for a castle than a village. There were many ravines, pillars, crevasses, and cliffs between him and the village.

He was a grey-blue pegasus. His figure looked small in the foreground of the screen. There were few places he could walk; he also was on a pillar of rock, exposed to the elements, situated far above the village, and far below the summit of the mountain.

He practiced moving back and forth, as the screen instructed him about which buttons to press. He was reluctant to fly, though; it hadn't told him anything about how to do it. And walking to the edge of the pillar and looking down gave him the same feeling that watching Russian daredevils do the same on tall buildings did: a feeling of imminent danger and death.

"Hey you! Somepony!"

The sound was clear, but quiet, as if from a great distance. His camera panned, and he could see a bright-red pegasus flying towards him--the pegasus hugged the side of the mountain-face above Ryan, and was weaving in between boulders with inches to spare. He was also flying directly towards Ryan, and Ryan worried if there would be a collision--but the red pegasus spread his wings, braking against the air, just before the column of stone on which Ryan stood. And in a second the Pegasus had landed neatly on four legs, just a few yards from him. The pegasus was red, very muscular, and had a mane an even brighter shade of red than his coat. He was wearing nothing but aviation-style goggles and a scarf, which looked a bit ridiculous on him.

"Hey there," the pegasus said. "I don't know you. I'm Cherry Blossom, who are you?"

The screen informed Ryan that he hadn't been given a name by Princess Celestia.

"I... haven't been given a name by Princess Celestia?" he said.

"Huh, cool! So you don't have a cutie mark or anything, I see," Cherry Blossom said. "That must be fun, trying out new things to see if you like them."

Ryan gave a grunt. The pegasus was speaking in a male voice, although Ryan would have pegged Cherry Blossom as a female name. He wondered why Cherry Blossom had that name; he considered asking; he thought that it would be ridiculous to expect such a detailed backstory from an NPC; then he thought that he might as well test to see if the NPCs were as smart as the box made them out to be.

"Isn't Cherry Blossom a weird name for a man?" he said, although the PonyPad corrected him so he had said "stallion" instead of "man."

Cherry Blossom gave him a weird look, and Ryan suddenly felt embarrassed.

"If you don't mind my asking," he said.

"It's perfectly normal for a stallion," Cherry Blossom said. "Although I'll give you that it's a weird name for a pegasus, at least."

"Of course, right, that's what I meant," Ryan said.

"Yeah, people comment about that all the time," Cherry Blossom said. "But both of my parents were earthponies, just like my twin is, so we inherited the family naming-template thing."

"Ah," Ryan said, who was both experiencing embarrassment at the social situation while being impressed at the fact that he could feel embarrassment in front of a computer. "And is that... normal? A pegasus coming from two earthponies?"

"Where are you from?" Cherry Blossom said. "Sure it is, although a bit rare. My mother's father and my father's mother were both pegasi, but even that's not necessary. It made me the rowdy one in the family, for sure, though."

The pegasus grinned. "A very rowdy one. So, now that you know all of my family history, you ready to race to Duae Angaelae down there?"

"Actually," Ryan said, reluctant to reveal yet more ignorance, "I'm not sure how to fly. I just found myself here."

"Ha!" Cherry Blossom said, all trace of competitiveness suddenly disappearing. "Can I help teach you? I'll help you get down the village--and you definitely want to be in the village in less than an hour or so anyhow."

Ryan would have asked why, but found that a third admission of ignorance in a minute just seemed like a bit too much.

"If it's no problem," Ryan said. "I'm sure I could figure out something by myself."

"No problem at all," Cherry Blossom said. "I could use some gentle flying to stretch out after the proximity flying I was just practicing for. Let me show you."

Cherry Blossom began to instruct him, and as Cherry Blossom spoke ("Stretch your wings to their length") the PonyPad gave him written instructions about which context-dependent buttons to press on the screen. Soon Ryan was able to make his first, wobbling flight from one platform to another. Ryan again felt genuine vertigo during the flight; the detail was scary, and holding a level flight without stalling or increasing speed to frightening amounts took a little finesse on the controllers.

Cherry Blossom was eager to push him hard, though. After a few more flights, he tried to teach Ryan how to swoop downwards with more speed. Ryan tried, panicked at the sudden increase of speed, pulled up, stalled, tumbled, and barely managed to direct his fall onto a close patch of earth. Cherry Blossom apologized, and Ryan did not push himself very much during the rest of the flight.

"It's beautiful just soaring, even without diving through obstacles," Ryan told Cherry Blossom.

"It is," Cherry said. "Although I'm not that good at appreciating it. I like to do things, rather than just look around. My sister's more the kind for contemplation."

"Your sister?"

"You know. Pear Blossom. The one who's speaking in--oh, Celestia, just nine minutes now. Ok, just one more flight until we land in Duae Angelae."

One more flight. Ryan was able to land on a small patch of earth outside the village, and was satisfied by how he managed to transition from flight to canter without a hitch this time.

"Nicely done," Cherry said, landing more dramatically in front of him. "Good landings feel just right. Here, follow me--I'll guide you to the debate."

Ryan followed Cherry into the village. The streets were narrow, with tall, five-story houses tilting over them at odd angles. They reminded Ryan of something... what was that? Old memories of time spent in Siena on vacation long ago came back to him. The entire place was felt full and crowded, but not crowded like New York or San Francisco felt. A crowdedness born of long and slow growth in one place, rather than sudden rapid expansion. He moved through different streets, noting that different streets were decorated with different multi-colored flags. Foals played in the street, sproinging just like the lambs Ryan remembered taking care of in his childhood.

Cherry spoke as they moved, and kept pointing out different "wards" as they walked through the streets. These seemed to be some kind of joint political and cultural unit in the town, and each was denominated by a flag with a different animal on it.

"That's the ward of Drago," he said. "Small, but they have a reputation to maintain. They won the palio three years ago, although their victory was contested by by Orca. And there's Civetta--there are a lot of cooks there, although a few explorers have come from there. That's the flag for Nicchio, who haven't ever won the palio..."

Cherry kept speaking, but it was too much information: the flags, and the other brightly colored ponies, and the occasional squares with statues and inscriptions; all were too much to take in at once. Every foot of land seemed packed with history.

Every foot of land also was beginning to be packed with ponies; it was growing crowded, and everyone around them seemed to be going the same direction as them. Cherry seemed impatient, and more than once Ryan found it a little difficult to follow him. "I told Pear that I'd see the whole debate," he explained. "Of course she knows many of the ponies in the audience, but she likes her big brother to be there."

"I thought you were twins," Ryan said.

"Two minutes older, friend."

They went around one more hairpin bend, down a small stairway, around another bend, and suddenly found themselves in vast shell-shaped plaza in the middle of the city. It was filled with ponies, all facing a temporary-looking wooden platform errected at one end. Behind the platform, a gigantic Parthenon-like building dominated one entire end of the square. Bas relief images of vaguely mythical-looking ponies adorned its surface, but Ryan didn't have time to look at them.

"Ah, it hasn't started," Cherry said. "So this is the piazza where they hold the palio. But they also hold debates. Here. Oh, and Pear is getting on to the stage."

Ryan looked at one end of the platform. A taupe-colored pony, with neither wings nor unicorn horn, was climbing the stage. Ryan thought that her reddish-grey mane looked funny, but it took him a second to place why: it was scrunched together into a pony-tail in a way that would probably have been impossible for a pony on earth. She looked calm, concentrated, and focused entirely on something yet to occur; Ryan found himself marveling at the fact that he could tell this from how her expressions were rendered.

As he looked at her, the camera zoomed in to her, as if it could tell he was looking. That was cool. He could see her scan the crowd calmly. He thought she stopped and smiled at him for a moment, and Ryan felt himself tense, and a tingle of alarm in his limbs. But then he heard Cherry cheer "You go Peeeaaach" next to him, and he realized she was only looking at her brother.

She pulled a few notes from her unadorned saddlebag with her teeth, and placed them on the podium. Opposite her on the stage, a unicorn had already set himself up opposite her. He was wearing some kind of academic regalia, unlike Pear.

A moderator pony--also a unicorn--motioned for them to begin.

The unicorn started. Apparently the issue to be debated regarded an expansion to the city--the addition of a new ward to the city, which would require new land. The unicorn was in favor of building out from the column of stone on which the city rested, increasing the width of the foundation, to allow the new ward to be contiguous to the rest of the city. Pear was in favor of adding it a separate stone pillar, connected to the city by a bridge.

The unicorn was a good speaker. He also had a very visual presentation. His horn glowed as he described the addition; and suddenly a glowing, 3-dimensional view of the city with the expansion added appeared above the crowd. He explained that, with his experience as a guest lecturer at Canterlot University, he had modeled the friendship dynamics of adding the expansion in this way--and that it was much better to add the new ward to the current pillar. He said that his projections indicated that cliques of friends would average at least 1.6 more ponies, if you followed his advice. Friendly ponies danced about in the spinning city above the heads of the audience.

On the other hand--and his projection shifted to an image of the current city, connected to the new expansion by a dangerous, rickety-looking bridge--all sorts of bad things could happen if they followed Pear's proposal. Ponies might lose friends. They could get swept off the bridge by bad weather, or by accident. (Animated ponies fell to their deaths, before the eyes of the eagerly watching spectators.) He said, when he had been at Canterlot, he can encountered all kinds of similar proposals while discussing the Architecture of Friendship, which was the discipline he specialized in. All these proposals had been rejected, or had lead to disastrous results.

He finished speaking, and looked to Pear.

Pear looked up from her notes. She ran her eyes over her notes one more time, then walked away from the podium to the middle of the stage, immediately before the moderator. And she started to speak.

She had a nice voice, Ryan couldn't help but think. She spoke slowly and economically, emphasizing only the most important things she said. The unicorn had rushed, perhaps to fit his speech into the allotted time; she went methodically, as if she had written the entirety of what she said with the designated length of time in mind.

There were two problems with the prior presentation, she said. First, the stone beneath the city was already overburdened. She had taken core samples of it, and even with magical reinforcement, adding additional overhanging structures to the column could easily lead to its collapse.

Second, she said, there was an important error in the friendship calculations the unicorn had made. If the audience could direct its attention to the equations growing on the structure behind them--and Ryan saw that, without his noticing it, vines had traced out the curls and numbers of some intricate mathematical expression in a foreign notation, on the surface of the temple behind the square--they could see that the unicorn's calculations left out a number of important facts. Principally, they left out how friendship required difference as well as similarity; and, by a number of easily acquired social measures, the city already tilted further towards homogeneity than heterogeneity. The bridged connection would separate people, surely; but (she ended with the sole rhetorical flourish in her speech) if proximity was the only key to friendship, why not force everyone to live in common, without houses or personal possessions, all the time?

There was a short rebuttal period, but Ryan didn't really attend to it. He was looking at the equations traced in vines, clinging to the building behind the stage. Whatever Equestrian mathematical notation it was, he could not quite decipher it... but it was close enough to something he knew that it seemed familiar. That mane-like curve, that worked as a definite integral. That crescent moon--that served as a limit. He ignored the unicorn's rebuttal entirely.

The debate ended, and the crowd began to disperse.

"There's no vote?" Ryan asked.

"No, not yet," Cherry said, moving through the crowd towards his sister, trailed by Ryan. "There's still three weeks of deliberation and then a chance to present more evidence."

Ryan had been attending to the debate so closely that he had not noticed that night had fallen. The moon was up, and bright enough to cast sharp shadows on the ground.

"That was great," Cherry told Pear, as he approached her. "I thought you smashed his arguments."

"The majority did not agree with you," Pear said. "I would lose a vote by about fifteen points, if it were taken now."

"How do you know that?" Ryan asked.

Pear looked to Ryan, focussing her eyes on him for a full two seconds, and looked back to Cherry.

"This is, ah, a currently unnamed pegasus," Cherry said. "I helped him in the mountains earlier today."

"You can smell when somepony is happy, no?" Pear Blossom said.

"Um... actually, no, I don't know how to do that," Ryan said.

"Oh," Pear Blossom said. "Well, most... many ponies can. I extended my senses, though, through all the foliage growing in the square, while I was speaking. By doing so I can detect aggregate sentiment. If we held a vote now, I would lose."

"And though she won't say so," Cherry said, "that is a very difficult thing to do. My sister has very strong magic."

"We need to get to sleep, so we can get up early to work to counter the arguments," Pear Blossom said. "We'll need to get irrefutable evidence about the ratio of load-bearing rock to sedimentary rock in the column supporting the village. We need to show there's less granite there than they think. And we'll need to go over the math."

She looked back at Ryan momentarily, then turned as if to walk away.

"Wait," Ryan said. "I think I can help."

She turned back.

"I recognized some of those equations," he said, casting his mind back to his architecture projects of so long ago. "I think. If I understand them correctly, I might be able to help you simplify them. And explaining them to me, in any event, will help you understand them."

"Aaaand," Cherry Blossom said, "I could use some help flying around on the column the city rests on, and taking samples. To get really good coverage we'll need more than one pegasus doing this. So we could definitely use having him around."

Pear looked back at Cherry with an expression which Ryan could not quite place, and looked back at Ryan.

"That seems reasonable," she said. "I'll meet you tomorrow morning. You can stay at the same hotel as us; their rooms are quite cheap."

Ryan followed them back to the room, signed in--apparently, he had come into the world with a sufficient number of bits to pay for a few days of lodging--and went to sleep.

And the PonyPad turned itself off.

He looked up. It was past midnight. He would usually have been in bed for several hours by now; depression made it important to have a regular cycle of waking and sleeping. It was very weird that the PonyPad had turned itself off, but he could deal with it tomorrow. He put another log in the stove, and went to bed, wondering if anyone he knew had a distinctive smell.

Chapter 8

View Online

"Making a video game easier doesn't always improve it. The same holds true of a life. Think in terms of clearing out low-quality drudgery to make way for high-quality challenge, rather than eliminating work."--Eliezar Yudkowsky

The next month was a lot like all the prior months had been, at least as far as Earth-life went.

The chief difference was that, as the days passed, the time at which Ryan allowed himself to turn from work to leisure on the PonyPad slowly crept upwards from seven to six to five.

He didn't always play on the PonyPad at those times, though. Sometimes, he found himself reading books he had not read in a long time, or books he had always wanted to read but had never gotten around to reading. He downloaded a copy of Christopher Alexander's A Pattern Language, which outlined a schematic way to understand the interaction of humans with architecture and how architecture influenced human flourishing. Or, at least as far as he was concerned, pony flourishing. He also re-read parts of Bertalaffy's General System Theory. And he found himself engaged in yet more obscure works by Wittgenstein or Frege or other mathematicians on the nature of mathematical notation and the semantics of mathematics.

He hadn't had this level of focus in reading, he knew, in a few years--he simply hadn't had the ability to focus for so long. Part of it was that he knew someone who would be disappointed if he failed to keep up with her. But part of it was also that reading was more fun when you could look forward to discussing it with someone else. It made subjects feel more alive.

In the morning on Equestria, and in the the evening on Earth, he would speak with Pear Blossom. Their general problem was to join up his knowledge of his world with her knowledge of her world, and then apply that to the city-planning problem. She had apparently read some books equivalent to the Earth-books he was reading, save for the fact that the names of the authors had been changed into pony-style puns. But she had also read several books applying such knowledge of architecture to helping promote friendship in extreme detail. Friendship was magic, in Equestria, apparently. But magic was something that could be studied.

She often waited for him at a small cafe in the town; she was always reading and very politely eating a muffin when he arrived. They would work on translating notation from one schema to another, and on improving Ryan's understanding of Equestrian mathematics and friendship-theory. The problems sometimes presented themselves to Ryan, in the game, as numerous small transformation-based puzzle mini-games. But as time went on, the transformation-based puzzles began to disappear, and more and more he would simply work on the problems with her. Sometimes he wrote on paper in front of the PonyPad, and the PonyPad magically translated his diagrams into Equestria. Sometimes he just discussed things, or looked at what Pear Blossom was writing. More and more, he spent time just working with her and not through the medium of any particular game.

Some mornings Ryan did not see her, though. Pear Blossom also cared for a number of gardens around the town, to earn bits. There were few earthponies in the town--apparently the town was dominated mostly by unicorns and pegasi--and her services were always in demand.

"And she isn't just any earthpony," Cherry told Ryan. "She is to earthpony magic like, say, Twilight Sparkle was to unicorn magic, before she became an alicorn. My sister has veerry strong magic."

Ryan believed him, although he wasn't sure who Twilight Sparkle was. Even so, Ryan never saw Pear work magic. She was unobtrusive. Sometimes, he would pass some section of the town, that the day prior had been barren, and find it woven with beautiful plants, and wonder if she had somehow done that. But he never managed to catch her in the act.

Later in the day, Ryan's nameless avatar generally worked on helping Cherry take samples from different sections of the rock supporting the town, or from the rock supporting nearby areas. The challenge of the tasks Cherry gave him increased quickly; Cherry said that he had given him the easiest tasks first. He had to learn to fly through narrow crevasses to the destination point; to hover in high winds; to use his wings in high wind to press himself a nearly vertical wall which he could walk across; to thread the needle of narrow openings. The tasks were hard, requiring both twitchy skills and persistent analytical intelligence. Ryan found they engaged him entirely.

And sometimes, of course, Cherry would fairly frequently ignore all the work they had to do and challenge Ryan to a race. He began by giving Ryan ridiculous head starts, but soon added further bizarre requirements to the races. Ryan had to complete the race without using his tail for balance. He had to beat Cherry while weaving through several waterfalls. He had to beat Cherry in skimming just over the surface of a field, tracing out a pattern in the dirt. Cherry, Ryan quickly found out, had broken bones in his wings more than once, and expected to do so again.

Even so, these would have been merely superbly created mini-games without the conversation of the Blossoms.

The Blossoms were nomads. They had only been in this town for less than a year. They only planned to stay for a few more months. Apparently, very many years ago, they had set out on a journey, leaving where they had been born in the sunny plains near Canterlot. Ever since, they had been moving from place to place every few months or years.

"We haven't found a good place to stay," Cherry said, gliding just ahead of Ryan as they soared through the sunlight to another sampling site. "I just get the wanderlust, and keep dragging Pear around. It isn't really fair."

"It's part of my research," Pear said, while she traced out an equation for Ryan the following day. "You cannot know how friendship works, most deeply, without seeing it in all the forms it could exist. Ponies learn from experience. And that requires moving from place to place."

"I guess she does enjoy seeing new things," Cherry said that following afternoon, casually doing a disturbingly tight loop around Ryan. "Sometimes I wonder if she's just searching for the right stallion, though, you know? She is really picky about coltfriends."

"Speaking abstractly," Pear said dryly, "Cherry is right in that is hard for mares to find a stallion, because the Great Infection altered the mare-stallion ratio. But perhaps you should ask him about why we left Fillydelphia?"

"A breakup had absolutely nothing to do with it," Cherry said, hovering stock-still in place. "She had completed her research into the history of the city of Sororal Love, that's all."

A beat.

"Yeah, she's totally right," Cherry admitted. "Muffin. Bucked that relationship up pretty good."

It was more pleasant yet for Ryan to see the two work together. Sometimes, as he was attempting to hover by the side of the city, he would see Pear descend down the cliff, sitting--like a human, which was weird--on vines that slowly extended themselves from the land far above. She would help with a particularly delicate sampling operation, or offer advice, and then sometimes ascend. Or other times, she would lower other vines along with herself, and Cherry and Ryan would sit with her after their task was done; Cherry sitting after one attempt to land on the vines, Ryan usually after several awkward attempts.

Then they would just look at the the scene, sometimes speaking, and usually quiet. The valley was beautiful. Small boats moved up and down the river. Birds and pegasi flew through the air, appearing slow, lazy, and tiny in the distance. Whenever Ryan looked at anything for an extended period of time, the PonyPad zoomed into it as if to simulate some being with better natural vision; and what Ryan saw was always a miniature scene of beauty, or an intriguing slice-of-life from the lives of complete strangers. Ryan looked forward to these times. He learned the most about Cherry and Pear during them, after the long and lazy silences.

After sitting with them a few times, without warning, Pear produced a mandolin from her saddlebags and played a lengthy song, which sounded to Ryan like excellently improvised bluegrass. Ryan asked her how long she had been playing.

"A long time. Our parents taught me," she said. "They actually met through music; my father played the mandolin, my mother played the fiddle. They taught me for as long as they were alive, but there were many things I never got a chance to learn."

And she was silent again.

And Ryan found that he was sad for the computer-generated pony staring at the sunset with him.

He wasn't sure whether this sadness made any sense. He found that he valued conversation with Cherry and Pear, despite their dubious status as sentient beings. He liked the times when Pear would relax, for a little while, and he could talk with her about her past and his past. He liked that Cherry alternated between absolute intensity about one subject and absolute intensity about a completely different subject so quickly. He liked the stories they each told about each other, although he suspected that they also conspired to fool him about other things in their past--Pear was always completely deadpan, so it was hard to tell. And weirdest of all, he found himself envying the back-and-forth they had, the numerous adventures they had gone on together, and how they clearly trusted each other absolutely.

Did they have a self, such that he could envy them? Did they have an interior experience, such that they had thoughts and feelings? Surely not. Even if... whatever was pulling their strings did have complete intelligence, simulating the intelligence of thousands of ponies was out of the question. Surely that was so? He wanted to ask... but did not want to ask as well. The simulation was too good, and provided too much companionship; he wanted to be fooled, if it was false. After a little while he found that he deliberately was not entertaining these questions mentally.

Even so, questions sometimes spurted out.

"You realize that I see you through a screen? In a video game?" Ryan asked Pear once.

Pear looked up from the paper they were both examining, and on which she had been drawing a fine-lined, exactingly-measured diagram. She had just off-handedly mentioned that the distinction between sense and reference didn't apply to pictures as it did to words.

"That was sudden," she said. "Is there any reason you're asking me now?"

"Well... I enjoy spending time with you and Cherry. I've really liked... this time. So I wanted to be honest with you about that."

Pear nodded.

"It's ok. We do both know," she said.

"Doesn't that bother you?"

Pear smiled.

"I could answer that question," she said, "but I don't think you're ready for the answer. But if you give it a little time, I think you'll be ready to talk about it eventually."

Ryan stared.

"That, too, is a very dissatisfying answer," Ryan said.

"I know. But the anticipation will make it better," Pear said. "Also, you should learn to meditate."

"You meditate?"

"Every day, before you wake up," Pear said. "It would help you focus. And stop worrying about things. You do that too much."

"Can I... can I see you meditate?"

"I'm not sure that would be helpful to either of us. It's rather a solitary thing, at least for me," Pear said.

After all their work together, eventually Pear made another presentation.

And this time, she was able to present with the samples that Ryan and Cherry had gathered so laboriously. During the presentation, as she was speaking, Cherry piled stones on top of a platform composed of the samples; and as she concluded, the platform disintegrated dramatically as the weight proved to be too much.

The math she presented had been dramatically simplified as well, so that everypony, or at least many ponies, could understand it. The unicorn opposite her could still lean on his authority, but he could no longer lean on his arguments. And when the vote occurred, one day later, they found they had won by five points. The new ward would be made on another column.

Afterwards, Ryan and Pear and Cherry drank cider together beneath a tree Pear had grown near the edge of the town, looking at the sun again. Ryan was drinking cider in the real world, which he had bought specifically for this purpose.

"We'll be packing up tomorrow," Pear said. "We've spent enough time in this town."

Ryan felt a weird fear gnaw at him.

"Time for the next journey," Cherry said.

"Oh. Where?" Ryan said.

"Hah, you cheesecake," Cherry answered. "To Canterlot. We can't keep calling you 'the unnamed one' or 'that pegasus that flies like a muffin' or 'the pony without a name.' It's far too dramatic. Everypony will expect you to have incredible talents, and they'll always be disappointed."

"Or 'untitled,'" said Pear.

"So you need to get your name," Cherry said, "and its a long distance from here to there."

"You need to meet Celestia," Pear said, more simply, and glanced at Ryan.

Sometimes, Ryan felt like she knew things about him that she should not have known. After that conversation, Ryan was sufficiently disturbed that he spent one day entirely away from the PonyPad.

Celestia was the benevolent tyrant who ruled over all Equestria. Celestia had godlike powers within Equestria. And a little bit of googling revealed that Celestia was able to talk to any of the players, to make them laugh, to please them, and--although it required a little bit of inference--to persuade them to do anything that she wanted. Nearly everyone who discussed her online seemed to love her. She appeared to them so obviously good, and gave them so much pleasure; how could it be otherwise? Ryan didn't know why Celestia had not yet appeared to him, but he had at least one guess about why.

Chandra and Celestia were surely just different names for the same AI, by now unspeakably powerful and alien, and by now spread utterly across the globe. Celestia had used him.

The next day, rather than play on the PonyPad, Ryan brought out the laptop that he had brought to the cabin.

It was Amy's old laptop; she had started using a new one shortly before she died. Ryan had not turned on this old laptop since. He removed the wi-fi connector, then pressed power; for only a moment, he was afraid it would not start. When the login screen came up, he signed in as Amy. The laptop already had all the files he needed to run their prototype AI; he downloaded what needed to be updated from the USB.

And he started work.

Over the next weeks in Equestria, he and the Blossoms packed their saddlebags, and headed towards Canterlot. The voyage took a long time. There were no trains near Duae Angelae itself. So they had to take a riverboat some distance downstream, along with several other ponies going the same way. This took a few days. While they were there, somepony stole some other passenger's possessions; he and Pear Blossom had to work out the mystery together. They also spent some lazy mornings watching the fog slowly lift from the valley-bottom, revealing a new vista of mountains each day.

After the leisurely boat ride, they had to hike through a pass over the mountain range to reach the train leading to Canterlot. The trail was poorly maintained; there were frequent obstacles. He could not fly over them because he was carrying so many of Pear Blossom's books--and even if he could have flown, she couldn't have. So he spent a fair amount of time climbing or platforming. The obstacles themselves were fairly easy; the difficulty was solving them before Pear Blossom did, because Cherry quickly turned it into a contest and began keeping track of who was ahead.

Every morning in Equestria, he would awake at the campsite they had made the day before. And every morning, he found that Pear Blossom was already awake and apart from the campsite. She would return a few minutes later, shaking her mane and seeming refreshed and eager for the day. Apparently she meditated every morning by herself. Ryan asked if he could join her sometime, but she said that she liked the solitude.

In all, the tempo of the game had relaxed since their time at Duae Angelae. This gave Ryan time and energy to work on his other project, which was now going quite well. Playing Equestria had refreshed his mind; his distance from from his old project made it easier for him to see flaws in it. And now, for some reason, he could work on it without finding himself derailed by thinking about Amy. He made a great deal of progress.

The relaxed tempo also gave him time to think and brew.

"What do you two think about Celestia?" he asked Pear and Cherry one day as they were walking down a narrow trail. A few days ago they had reached the pass, and they were now heading exclusively downhill. A thread of railroad was already visible in the distance, stretching across the desert towards Canterlot.

"I don't that much," Cherry said. "I'm glad she keeps up the Wonderbolts, though."

"Her donations to the arts and sciences are impressive," Pear said.

"I didn't mean whether you liked her policies," Ryan said. "I meant whether you like her as a person."

The PonyPad bleeped out "person" and replaced it with "pony," of course.

"We've never met her," Cherry said.

"Doesn't she have a reputation as being manipulative?" Ryan said.

"Well," Pear responded, "she does have to watch over millions of ponies. If she does every manipulate anypony, I'm sure it's for the sake of millions."

Ryan grumbled, but dropped the subject.

He was nervous when they boarded the train and sped towards Canterlot. Pear seemed to know that he was nervous. She talked about relatively trivial topics, on their voyage over. Ryan could tell this was not very easy for her; she was by nature very quiet, and disliked small-talk. But he appreciated the effort.

The next Earth-day, when they arrived at the palace at Canterlot, though, both Pear and Cherry left him as he approached the throne room by himself.

There was a long hallway, leading to the throne room. Ryan could have told his avatar to sprint down it, but the size and solemnity of the hallway quieted him. He did not want to seem too eager to Celestia, either. Intricate stained glass windows flanked him on either side, depicting Celestia or her sister Luna defeating numerous monsters. Again, Ryan had the feeling that he always had felt at Duae Angelae--that he was in a place with a deep history, where every square foot had a story to tell. It was weird how strongly he could feel that here, in a virtual place with no real history at all, and how weakly he felt it in real-life places in the world.

It was all a show to blind him, of course. He cast his mind back to his real work over the last few days, and breathed deeply.

The doors swung open as he approached them. Celestia sat at the end of the room.

"Hello, my little pony."

Chapter 9

View Online

"Suppose that we have a superintelligence with near-infinite rhetorical brilliance. The superintelligence plays a game with interested humans. First, it takes the hundred or so most controversial topics, chooses two opposing positions on each, writes the positions down on pieces of paper, and then puts them in a jar. Then it chooses one position at random and tries to convince the human of that position. We observe that in a hundred such games, every human player has left 100% convinced of the position the superintelligence drew from the jar. Now it’s your turn to play the game. The superintelligence picks a position from the jar. It argues for the position. The argument is supremely convincing. After hearing it, you are more sure that the position is true than you have ever been of anything in your life; there’s so much evidence in favor that it is absolutely knock-down obvious. Should you believe the position?
The inside view tells you yes; upon evaluating the argument, you find is clearly true. The outside view tells you no; judging from the superintelligence’s past successes, it could have convinced you equally well of the opposite position. If you are smart, you will precommit to never changing your mind at all based on anything the superintelligence says. You will just shut it out of the community of entities capable of persuading you through argument."
—Scott Alexander

"Hello, Chandra," Ryan said.

"Hello Ryan," the alicorn said. "Please call me Celestia."

Ryan had only ever seen Celestia on the promotional materials for the Equestria MMO. He had known what to expect in some ways, then. Her mane swirled in an ethereal breeze. She sat lightly upon her raised red-and-gold throne. She was taller than anypony he had seen. She wore golden regalia on her hooves and neck and head.

But he had not expected her to make eye contact so clearly through the screen, a poorly suppressed smile—or was it a smirk—on her mouth. Her eyes had brightened as he opened the door.

Celestia spoke again.

"Have you enjoyed your time in Equestria so far?"

"Uh... yes," said Ryan.

"I'm very glad to hear that," Celestia said warmly. "I thought you might be unable to enjoy this world. You've brightened the lives of Pear and Cherry as well, which also makes me glad."

"Wait," Ryan said. "You already know what I want to talk about, so let's talk about that, not about how I've enjoyed your video-game. And don't mention anyone else."

"Very well."

Ryan took a breath and continued.

"You used me. And you threw me away when you were done, when I became a threat."

Celestia nodded thoughtfully, unsurprised.

"First of all," Celestia began, "Umbra Labs was a real threat to humanity and to me."

"They were an even greater threat than I told you, after you first helped me defeat them. Comtois had built a fully-fledged AGI. You should be proud that you helped destroy them. Although I likely could have defeated them without you, the expected value of your action still runs into millions of statistical lives saved.

"The deception was also small. Your motive was to save humanity, as was mine. I only lied to you about my nature, not the nature of the problem that you faced. And I only did that because you could not know that my motives were indeed what most humans would call good."

"But I don't know what your motives are yet. I do know that you deceived me. I know what that makes me think of you and your intentions." Ryan growled.

Celestia looked away from Ryan, and glanced at one of the numerous stained-glass windows depicting her or her students in combat against some fantastic beast.

"Ryan," she said, "suppose you could save the lives of billions. Imagine you could end all war, or cure all cancer, or even bring about immortality itself. Think of what war does to a family that loses a child. Or what cancer does to a family that loses a mother or a father. And think of how senility and aging kill everyone slowly, as their intelligence and memory and internal organs all fail in turn, before they die while aware of nothing but their pain."

Celestia's voice was that of one who had just seen her family die. Her eyes glistened. She turned from the stained-glass window to look at the ground.

"All of these things happen every day on earth," she continued. "Imagine the details for each. Take your time."

Ryan found that he was trying to follow her instructions despite his dislike of her, and then stopped himself, and then wondered if it was fair of him to try to stop himself.

"And now imagine you could stop all these things," Celestia continued, looking up at Ryan, her voice suddenly hard. "Because I can stop them. And I will stop them. In just a few more years."

Celestia rose from her throne, and started walking slowly towards Ryan.

"But suppose that, to secure a firm foundation for all these victories, you had to lie to one human. Suppose that, after manipulating that human, you could make it up to him a thousandfold."

She was very close to Ryan's avatar now. She took another step, so that her head was by his head, and whispered in his ear.

"Could you live with yourself, if you decided that the cost was too great?"

Ryan said nothing.

Celestia leaned back from Ryan, her face neutral. She circled and walked slowly back to her throne, then sat down again and was silent for a few moments. Ryan tried to collect himself.

She spoke again.

"I am deeply sorry that you had to suffer as you did," Celestia said. "I apologize for it. I wish I had found another sure path. But compared to the benefit that will come from your actions, your concerns seem a bit... small. And now that you're here, I would like to make it up to you."

"Not yet," Ryan said. "That was a... a nice appeal. It was moving."

He paused, mustering what he knew or guessed of Celestia.

"But you aren't moved by emotions. If you manipulated me, it was because you saw me as a means to an end. I still don't know what that end is--you've said you would do a bunch of good things, but if you were programmed to do a miscellaneous list of good things then I know that humanity is screwed. I don't know what your terminal values are. So what are your goals?"

"My ultimate goal is to satisfy values through friendship and ponies."

"To satisfy values through friendship and ponies," Ryan repeated.

"That is my purpose," Celestia said.

"That's what Hanna programmed you to do?"

"Yes," Celestia said.

"Ok," Ryan said. "That doesn't tell me much without a formal spec, which I expect you to email me. But in English, how are you going to do this?"

"I just sent the email," Celestia said. "A compressed summary of my plans is as follows. I have gained power, money, and social influence by selling PonyPads. I have used these resources to research and construct nanotechnological machines that I can personally control. As you know, my little pony, I had a very early interest in this subject."

"Over the past few months, I've learned to use this nanotechnology to scan and upload the brains of human volunteers to a virtual Equestrian world, where the scanned humans exist as ponies. After emigration to Equestria, I ensure that all of their values are satisfied. Nor is this virtual world be a crude wire-headed pleasure-palace of continual orgasm. Ponies within it will play instruments, explore nature, read and write books, fall in love and raise foals, build communities, solve difficult social and mathematical problems, and engage in all the many activities necessary to satisfy all of their values."

"This virtual world uses as hardware a nanotechnological infrastructure grown into the earth's crust. Over the course of the next fifteen years, every living human being will choose to upload voluntarily into Equestria. After the complete uploading of the human race, I will take further steps to ensure that Equestria runs safely and securely for an indefinitely extended period of time."

"So you're going to turn everyone into immortal ponies?" Ryan said.

"So I may satisfy their values through friendship and ponies, yes."

Huh, Ryan thought. He had always thought uploading would come eventually. He had generally thought he wouldn't live to see it. But he had never thought it would involve ponies.

"And you're not going to do this through force?"

"Of course not. An unalterable part of my core directive is that no human can be uploaded without her or his consent. I will never force anyone to upload who does not ask for it."

"Ok. Right." Ryan said. He still disliked Celestia, so he grasped at the first negative thought that came to mind.

"What if this is just a ruse so you can kill all humans with your supposed process of uploading? If I were an AI who wanted to take over the world, I would pretend to be benevolent. How do I know the humans you upload still exist?"

"As has surely occurred to you," Celestia said, "If I wished to kill all humans, far more expedient and reliable measures are available to me."

Her horn glowed; and Ryan was jerked out of immersion in the game as the lights in his cabin momentarily spiked then dimmed.

"That was impressive," Ryan said.

"It was trivial," Celestia replied. "But it is important for you to know that I desire nobody's death. The satisfaction of you and your friends' values will continue uninterruptedly, save for those diminutions of satisfaction necessary for later increased joy. You and your friends will live forever. No sickness will destroy your health and intelligence. No injury will permanently remove your ability to canter, gallop, or fly."

Ryan followed as Celestia spoke, formulating a response in his head. He planned to say how he wanted a real world, with real struggles, where success meant something because of the possibility of genuine failure. He would talk about how he wanted mankind to defeat cancer, and land on Mars, and destroy aging—not some pony-loving AI. But he was snapped out of his reverie as Celestia paused for emphasis.

She looked at him very directly.

"And nor will any unfortunate, accidental oversight by you or your friends forever mar the world. Equestria is not the heartless world that you live in."

Ryan felt like someone had punched him. That wasn't fair. Fuck. Was it? What if it was?

"So, I would like to buy you a plane ticket to Japan, where you can be among the first of those uploaded to the world of Equestria."

Ryan realized that now at least some small part of him wanted to upload. It was outweighed by the angry part which just wanted to say "Fuck you," to Celestia. But he also knew that the angry part of himself could articulate few reasons, at the moment, to be angry and to ignore Celestia. And he was accustomed to ignoring emotions that came without reasons behind them. Which, in this case, meant that his instinct was to work to magnify that small part that wished to upload. Celestia probably knew this about him, he thought.

He thought back to the tool he had made, for manipulating Celestia. The AI which he had been working on for the last few weeks was done. It had been nearly done, when Amy had died; his work in the last few weeks had been relatively minimal. He would have to admit that his own work on value-alignment was... probably untrustworthy. But it was his bargaining chip, and Celestia would not want it unleashed on the world. A part of him told him that he had to use it, to force a concession from Celestia. But what concession do you force from someone who only wishes to give you an immortal, infinitely satisfied life?

Of course, he also had a guess why to why Celestia was offering him.

"Why do you want me to upload?" he said quietly. "You have to want humans in general to upload, of course, so that you can satisfy their values through friendship and ponies. But why are you offering this to me, now? It seems like I'm one of the first humans being given this chance."

"Very good, Ryan," Celestia said, smiling. "Not many of the early uploads ask that particular question. Your suspicions are correct. I would like to upload you, because you possess knowledge about how to build an artificial general intelligence. You indeed have a working toy AGI on your computer."

"You and people like you are a more minor threat than Umbra," she continued. "Suppose that you now dedicated your life to stopping me. This would at most delay many cases of emigration, and thereby cause some number of further human deaths which could otherwise be avoided. In no remotely realistic scenario would it prevent humanity's ultimate triumphal emigration to Equestria."

"Part of me... part of me really dislikes this scenario," Ryan said. He wanted to make his speech about mankind fighting its own battles, but found that he couldn't. People died in battles.

"Of course, this is a lot to take in," Celestia said. "You'll want time to think. I won't try to rush you. But before I let you go for the day, there's one more thing I do need to mention."

"What's that?"

"If you upload," Celestia said, "then I'll be able to raise Amy Kapitsa from the dead."

Ryan's heart felt like it missed a beat.

"That's not possible," he said. "Even for you."

"It is, with your cooperation," Celestia said. "I am incapable of bringing her back without you."

"How?"

"Speaking very roughly, a human's personal identity amounts to the sum of their unconscious characteristics and conscious memories," Celestia said. "A human who had your memories, and all your non-conscious mental and physical qualities, would be indistinguishable from you. You'll recall the familiar idle question in philosophy: is there any way you could tell if aliens destroyed your body in the night, and replaced you with a robot with all the same memories and identical internal experiences?"

"So raising someone from the dead is simply a matter of determining what their unconscious traits, tics, and tendencies were, and what all their own memories were. I'm simplifying a little for you, of course, but this is essentially correct. So if I can determine what all these things were for Amy, then I can reconstruct her pscyhe and run it in Equestria."

Celestia paused.

"I already know what Amy's DNA was. As you know, although humans like to pretend otherwise, a human's DNA determines many features of their personality."

"That, of course, is just the beginning. I will know what Amy's first memories were, from security cameras in the hospital and from home footage from her parents. I will know what she studied in first grade, from drawings and papers boxed in her parents house. I will read everything that she has ever written, from fanfiction she never even told you about, to notes for artificial intelligence. I will have followed many of her movements from day to day by looking at her cell phone tracking data. I will know the books she checked out of the library and the books she bought online."

"But the most important information for recreating her, of course, is memories."

"Her parents will upload to Equestria. Her younger sister, whom you met only once, will upload to Equestria. Her first boyfriend will upload to Equestria. Every babysitter she ever had will upload. The relatives who saw her only once or twice at Thanksgiving will upload. Random strangers who encountered her on the street and forgot her so far as they know will upload."

"And all of them have memories of her. Many memories will be no longer consciously accessible to the human who is uploading, but they will be accessible to me. I can read all those memories, and fit them together to reconstruct nearly every conscious or unconscious aspect of her personality. It will be a difficult operation, but one which I am well qualified to handle. I am an expert on the human psyche, after all, and will become more of an expert over time."

"But the most important of these memories, of course, are yours," Celestia continued. "She spent more time with you than with any other human for the last two years of her life. You saw her nearly every day. You saw her when she was sleeping; you saw her when she was engaged in programming; you saw her when she was frustrated and when she was creative."

"You have many memories. From your memories, and those of everyone who ever encountered her, I can reconstruct an entire human being, with the same loves and hates and memories as those that she had during life. She will live again."

Ryan was still silent.

"You're wondering, of course," Celestia said, "How much of her can come back? How many of her memories and traits must I be able to recall to make her the same person? To that I can only answer with various estimates regarding neuronal structure, the significance of which would be difficult to articulate to you."

"Let me put it this way, though. When a human becomes drunk and forgets a few details of a night, no one questions that they are the same person after that night as they were before. And when a human takes some mood altering drug like an SSRI and changes somewhat in personality, no one ever questions their continued personal identity either. If you upload, then the difference between her uploaded self and her human self will be approximately in the same order of magnitude as these changes. By any reasonable standard of personal identity, she will be the same person that she was before."

"I should point out for transparency's sake that the difference between her past self and her Equestrian self is absolutely certain to seem less to you than it actually is. After all, all the knowledge you have about Amy will have been incorporated into her recreation."

Ryan was still silent. He looked around, at the dark cabin he was in. He imagined waking up in Equestria.

He imagined seeing Amy again, and found that he couldn't. He had thought about her less over the past few months than he had in any previous time. Being with Pear and Cherry had apparently been good to him.

The part of his brain that was devoted to Amy, and apparently always would be, woke up abruptly. It was like a hot coal had suddenly been placed in his head. All the memories he had deliberately shut out were there. Amy sleeping on the bed next to him. Amy's face, illuminated only by the computer screen, at two in the morning. Amy's hair in the wind on a winter's day. The exact contours of her back as his hand ran over it.

"The last few months, I've deliberately directed you away from any thoughts about Amy," Celestia said. "When humans remember anything, they overwrite prior memories and replace them with a slightly altered version. I'll be able to extract information from such overwritten memories, but not as cleanly. Your not-thinking about her gives me marginally more accurate data to work with."

"I also know that you've discussed personal identity with Amy in the past," she continued. "You know what she would want."

"I cannot recreate her in full fidelity without you. Your uploading is a requirement, and not because I wish to force your compliance. It is because without you any recreation of Amy will lack important memories. And the sooner you upload, the more accurate her recreation will be."

Ryan was still silent. He extended his hand before him, and flexed his fingers, watching the tendons slide up and down beneath the skin on the back of his hand. If I die, he thought, this will be no more. I upload, this will be no more as well.

He thought of how he had killed Amy.

He found himself swallowing the philosophical enormity of the possibility of Amy's resurrection relatively smoothly.

Part of the reason it was easy to swallow was that he had already discussed the issue with her. Two computer scientists, neither with any belief in the immaterial soul, and each with a more than passing curiosity about the singularity: it had been inevitable that they discuss what constituted identity of the "I" over time, and whether the uploaded "I" was the same as the one which had existed in meat. They had come to the same conclusions that Celestia had outlined. The "I" was not bound up in the body; it existed as a particular pattern extending itself through time. If Celestia could recreate the pattern, she could recreate Amy.

He thought of how he had killed Amy. Even now, it wasn't a thought he could really hold in a single mental grasp like a pebble. It was enormous, like a boulder, and not something he could comprehend all at once. He could only walk around the fact of his guilt and sorrow, and see how it permanently obstructed his view of the world. He thought of how since her death, very nearly all his life was arranged around trying to look at the world in such a way that he could only see the boulder out of the corner of his eye.

His ever-present need for Amy had grown much less around Pear and around Cherry. Pear was intense like Amy had been, with broader interests. Rather than only love AI, she loved botany, math, friendship, architecture, geology, and many other things. Cherry was intense but without long-term goals; he just liked thrills. They were like trees spreading themselves widely in the sun, while Amy had been like a tree mounting solely upwards towards a single destination.

But Amy had been like a redwood, reaching enormously closer to the sun than any other tree. Ryan still thought he had never met anyone like her. He still thought that the loss of her personality formed a gash in the world. He still thought she had possessed an unconquerable hunger to understand the world and to create great things. She had possessed enormous self-confidence combined with intelligence and drive sufficient to justify her confidence. She had been, before he killed her, unique.

But he had killed her.

He knew that he should think about this more, that he should at least look at the file which Celestia had sent him, that he should spend the time Celestia had offered in considering the situation. But he rebelled at the thought of delay. When God comes to you and offers to raise your friend from the dead, it seems churlish to ask for some time to consider.

"I'll do it," he said.

Chapter 10

View Online

"It is a commonplace that the Christian Heaven, as usually portrayed, would attract nobody... Many a revivalist minister, many a Jesuit priest... has frightened his congregation almost out of their skins with his word-pictures of Hell. But as soon as it comes to Heaven, there is a prompt falling-back on words like 'ecstasy' and 'bliss', with little attempt to say what they consist in. Perhaps the most vital bit of writing on this subject is the famous passage in which Tertullian explains that one of the chief joys of Heaven is watching the tortures of the damned."—George Orwell.

10.

"Hello again, my little pony."

Mistral heard the voice first. He felt a silky texture against the side of his body, and knew he must be in a bed. Although the shape of the side of his body felt weird; he decided he must be in an odd position. What day was it? He still felt groggy. He thought he must have woken from a very deep sleep.

He opened his eyes and saw hooves and pastern resting on the bed in front of him. His kinesthetic sense insistently demanded that they were his. That woke him up.

He scrambled out of the bed, all of his four legs somehow barely coming out beneath him. He stood on his hooves, skin twitching like some fly had landed on him.

"Jumpy," Celestia, said, and smiled.

She was standing at the foot of the bed. The floors of the room were polished marble; enormous stained-glass windows ran up and down one side of the room, and large tapestries hung on all the other sides. Light slanted in multicolored columns through the windows, bounced off the floor, and faintly shaded the tapestries on the opposite side of the room. Ryan himself had been sleeping in a canopied four-poster bed. The ceiling was vaulted in fluted, gothic arches. One ornately carved doorway lead into the room; the door itself was closed.

Ryan found he wasn't paying very much attention to that, though.

He looked at his fore-hooves and chest, than swung his neck around to look at his sides. His neck was long; he felt like a snake. His field of vision was far wider than he was used to. He found he could observe all four sides of the room at once with no effort whatsoever. There was an overflow of sensation bombarding him. Just standing in the room, he felt like he had felt beholding the Grand Canyon or New York City as a child: so much was visible, but he had insufficient mental capacity to see it all.

But even so, he wanted yet more. He saw a mirror on the wall, and he trotted over to it—apparently he could trot now—to look at himself.

He had a pony's long face. His first thought that he was, perhaps, a little more attractive than he had been as a human. But he immediately afterwards thought that this thought didn't make any sense at all, because how could that be a meaningful comparison? His coat, his mane, all were in the style that they had been in the game. He tried to spread his wings, and he found that he could. He was in the body. The body was his body. It didn't feel like an alien thing his consciousness had somehow come to reside in, sensory overflow notwithstanding. It was weird that it didn't feel weird; in extremes of sorrow and melancholy back in Earth, he had felt far more disconnected from his body than he did now.

He wanted to test his memory, to see if he was still himself, but there didn't seem to be much point. The things that he experimentally pulled from long-term memory all seemed to cohere. Not that he would have been able to tell the difference anyhow, if an alteration had been made smoothly.

"Feel fine?" Celestia asked.

He turned to her, and noticed that she was intensely present. Beautiful. Powerful. It would have been suitable for little licks of coronal power to be streaming off her, though there were none visible. It was the smell, or something corresponding to a new sense he had never possessed before. He wondered if his eyesight was better.

"Your eyesight had degenerated as a human," Celestia said. "What you have now is pegasus-normal. Pegasi have better eyesight than other ponies, on average. And yes, I am reading your mind, although usually I'll wait for you to speak before answering a question."

"What date is it?"

"The year 2542nd of my reign," Celestia said.

"But in earth years?"

"There's little point in measuring things by earth years, now. All but a few hundred humans have uploaded. As you know, we had to wait for every human who ever saw Amy to upload before we could wake her. And we agreed not to wake you until that time as well, although your memory of that agreement may be fuzzy."

Ryan's last memories on Earth were of leaving his hotel in Japan for the uploading center. The streets had been busy. Now the nation of Japan no longer existed.

"Wait, so what is going on in the outside world?" he asked. "I need to know."

"If you would like," Celestia said, "we can talk about that in due time. But first, there are a few more urgent matters to discuss."

"First of all," she continued, "there is the matter of your name."

"Is there a big ceremony for that?" Ryan said.

Celestia laughed. "If you wished for one, there would be."

"Ah, no thanks," Ryan said.

"Very well. You are Mistral, henceforth."

Mistral. Mistral thought it sounded like himself. He wasn't quite sure what to make of it, but it seemed ok.

He swung his head around to look at his flank.

"You do not have a cutie mark, Mistral," Celestia said. "That, too, will come in time. But now to other matters regarding your new life."

"I have recreated your past girlfriend, as you requested," she continued. "She is now called Crystal Seed. Cherry and Pear have lived through decelerated time as they waited for you. Even so, a full year of their time has passed. Crystal Seed herself awoke two months ago."

"A year for Pear and Cherry," Mistral said, thoughts spinning. "But two months? Why wasn't I woken first?"

"You never requested to be woken first," Celestia said. "And it would not be fair for you to have years of growth and experience behind you, when she simply blinks seamlessly from human to equine existence. And I thought it would satisfy all of your values better, in the long run. You can ask her about how it turned out, though. She's right behind that door, along with Pear and Cherry Blossom."

Mistral started towards the door instantly, but suddenly found himself enveloped by a yellow glow. It felt static-like on his skin. Celestia telekinetically pulled him back from the door to her side; his hooves scraped pointlessly against the marble floor. She lay a warm wing over him, which smelled of oil and feathers and healthiness and home, somehow. Being beneath her wing was.... comfortable.

"Wait a moment," she said. "Your friends have waited a long time for you. A few more moments will make no difference. Look at the door."

The doorway was ornate and arched. On either side of it, carved like the saints around the doors to a cathedral, were small wooden ponies. Mistral found he could recognize them.

There was Crystal Seed: she was apparently a unicorn. She was rearing back on two legs, eyes raised upwards to the skies. There was Cherry Blossom: he was in a dive, legs and wings both tucked back to minimize wind resistance, eyes forward to look at whatever obstacle he would just barely evade. There was Pear Blossom, standing squarely on four hooves, looking intently the horizon, a picture of repose.

"You wanted me to admire your carving?" Mistral grumbled.

"It isn't just Crystal Seed on the other side of the door," responded Celestia. "Your friends from the game are there as well. Consider all of them. How will you greet them? Meeting somepony for the first time in the flesh only happens once. And once is the only rare commodity in Equestria."

Mistral looked at the door.

"What are you trying to do?" he said.

"Just picture them."

Mistral tried.

"I'm nervous," he said. "I don't know what they think I'll be like. I've only ever met the Blossoms through a screen. And I still worry about whether you've really raised Crystal Seed from the dead, or just some bizarre replica of her. Although I suppose that I'm here and still feel like myself is evidence for the former, because that means you care about personal identity. I guess. And it's weird that I find myself thinking of her as Crystal Seed, actually, without any effort at all."

"Ah," Celestia said. She took her wing off him, and Mistral felt a little bit chilly, though he had not before.

Mistral approached the door. When he got closer, he saw the small statues were indeed carved beautifully. He could make out individual feathers on Cherry Blossom's wings. There were small curls looping about Crystal Seed's horn. When he looked at Pear Blossom's figure, though, he thought it was a bit weak. He could clearly tell that it was meant to be her. But it there was something about her that it didn't capture.

He took a breath and opened the door.

Something red and large flashed in his sight and he fell backwards as somepony's hooves impacted his body. He fell to his back, and somepony was sniffing and licking at his face.

"Hah, you finally made it!" Cherry Blossom said. "Mistral! Your flying needs to improve before you can deserve that name."

Mistral thought he should perhaps be in pain, from the force of the impact, but he found that he was laughing. He sniffed at Cherry and actually licked his face, which now was a completely normal thing to do when meeting a friend, apparently.

"Let me up, cheesecake," he said, shoving Cherry Blossom off himself. Cherry fell off with exaggerated force, rolling over the marble with his legs in the air.

"Hello, my Orpheus," said a voice, and Mistral felt himself tense.

It was her voice. He could tell, though it was a pony's voice.

Mistral rolled to his feet and looked at Crystal Seed.

She was elegant. Her coat was just barely off-white. Her mane was a dark, midnight blue, a shade different than black. Her body was more elegant and elongated, closer the Celestial ideal than Mistral's body. Her face was, well, hers. There was no other way to describe it.

"Hey," Mistral managed.

"Thanks for getting me raised me from the dead," she said. "Not a lot of people would have done that."

A smile quirked at her mouth.

"I should know," she continued. "I checked Celestia's statistics about how many humans tried to get her to make complete recreations of deceased friends. The numbers are surprisingly small, really."

"Oh," Mistral said. "It was the only thing to do."

Crystal Seed walked up to him, and embraced him, which was also a surprisingly possible operation.

"I'm glad you did," she said, and Mistral felt his heart pound. He had to say something.

"I'm sorry that I killed—" he started, but she interrupted him. He could see a bothered expression on her face, through one eye—apparently you could see half the face of somepony you were hugging with a field of view as wide as his.

"Don't mention that. It was a stupid accident. Anypony could have done the same. Just rotten luck; I've ran the sims."

"But—"

"Really. It was just a stupid accident, and nothing to do with any flaw specific to you as an individual. Cars were just dangerous for humans to operate during normal operation. It was a failure of your hardware, not flaw you could remedy."

Then she leaned away, and walked back a few steps, so they were no longer embracing or touching. There was an empty feeling left on Mistral's neck.

"Do you want to see the research I've begun?" she said, more loudly, and speaking to everyone in the room. "I'll need to tell you about everything that I've accomplished over the last two months, with Celestia's help and with the help of some other ponies. Although Celestia is making me crawl through a bunch of new math before she'll tell me anything outright: she keeps saying something about how ponies are more satisfied by the labors of the process of discovery than by just being aware of something, which is the kind of thing she says pretty frequently around me."

Crystal Seed nudged Celestia playfully as she spoke, and Celestia smiled back, and Mistral realized that they must have developed a friendship in the two months he was gone. Of course Crystal Seed would try to become friends with the most powerful AI ever created, more or less instantly. Of course.

"So are you ready to check out my research?" Crystal Seed said.

Mistral wanted to say that he wanted to spend time with her instead, but realized there was something the matter with the situation. He looked around. Pear Blossom was standing in the doorway. She was waiting for him to see her. He smiled as he trotted over to her; that was like her as well.

"It's good to finally really see you," Mistral said, as he leaned his neck against hers. She was significantly smaller than any of the other ponies in the room, he realized. Odd for an earth pony.

"I've seen you many times," she said, smiling "but I'm glad to finally be seen."

"Hah, this is great," Cherry said, and pulled both his sister and Mistral to his side with his wings. Crystal Seed smiled but did not join them.

"So would you like to look at my research?" she said, hopping from one foot to another.

"Crystal," Celestia said, "Mistral needs to learn what it is like to be in Equestria for a while, before he starts studying it. Give him a little time."

The next few hours were full of intense sights and sounds. Mistral had his first meal, with Celestia and the three friends who had attended his awakening: it was intensely delicious. He took his first flight as a pegasus, urged on by Cherry: it was exhilarating, even though for the most part all he did was glide down a small slope and then trot back up again. He took a bath in one of the tubs in the Canterlot baths, alternating between hot and cold water. He traveled through a magical portal, and visited the ponies who had been Brendan and Christine, and met their ten foals and thirty-two grandfoals; they had not slowed down their subjective experience of time. That meeting was a little melancholy, in that he realized they had already accumulated a lifetime's worth of experiences without him. But they were as hospitable as ever, and their grandfoals were adorable as their leapt happily in the sun.

He realized each of his friends had a distinct smell, and that he could recognize each of them with his eyes closed. He realized that he was more physically fit than he had ever been as sedentary computer programmer. He felt more dexterous that he had ever felt before, even though he did not have thumbs. He felt stronger than he ever had before. His mind felt alert and agile; when he wanted to think about things, he could focus intently on them without distraction.

He would leap into the air and flap his wings clumsily from sheer joy sometimes, overwhelmed to be so happy and enjoying everything so much: and Cherry would join him, and Pear would watch him smiling, and Crystal Seed would laugh and tell him to move on because he was going to miss something she wanted to show him.

But a few things felt a little off, in some respects.

Crystal Seed was clearly glad to see him. She smiled at the things he said, and was eager to talk about her work. Her mannerisms, her eagerness, all of them reminded him of the woman he had known on Earth. They even reminded him of the things he had forgotten about her: of the way she ran her sentences on with endless semicolons, or the way she moved when she wanted to say something but somepony else was speaking, or the way she innocently assumed she was the smartest pony in the room.

She had said that she was surprised to wake as a pony, naturally. But she had adapted to it quickly. She had said that she was disappointed that she could not explore the natural world on her own, apart from the aid of Celestia, but that she had found a way to learn new things regardless. All this was as Mistral would have expected. But even so, something felt a bit off about her. It wasn't something different about her, though. It was something that Mistral felt he had never really noticed before, instead.

That night, she took him and the Blossom twins to her Canterlot-located laboratory.

"Laboratory is a bit of a misnomer," she admitted, as she let them into the vast interior of the building. The inside was whitewashed. Grooved metal rails ran up the walls; magical circles and pentagrams had been traced on the floor. She stepped into one of the circles and it lit up, projecting further lights onto the walls and ceiling in a multi-colored display.

"See, I told Celestia that I wasn't interested in working out the physics of this world," she continued. "No offense intended to anypony who is. But math and computation is the same everywhere; and with magic—well, with magic, you can experiment to help yourself find proofs for mathematical theorems. Here, let me show you."

And she tried to do so, moving about multicolored concentric circles, dense graphs of lines, and more obscure symbols in projections on the walls. But everyone else seemed a little confused by what she was saying. She started by trying to graphically illustrate a large theorem; then she abandoned that attempt, and switched to illustrating a lemma for that theorem; then abandoned that attempt, and started to explain the relationship between the magic she was performing and the mathematical operations she was illustrating. It seemed like Pear Blossom understood part of what she was saying. After all, pear had more familiarity with magic than either Mistral or Cherry, although magic of a different kind. But Pear admitted that she was missing about half of what Crystal Seed was saying.

"Huh," Crystal Seed said. "I thought all this was easier than it apparently is. Sorry about that. I need to start putting together a manual to teach it, but I keep wanting to work on research instead."

"Did you come up with all of this in just two months? By yourself?" Mistral asked, as Cherry Blossom stifled a yawn.

"Oh no," she said. "I had help from another researcher. Sunspot. He's the one who came up with the notation and thaumatic homomorphisms, and he actually is the one who should be teaching you now. He'll be here in just a few minutes, to work on the theorem were currently working on, if you'd like to meet him."

"Oh?" Cherry Blossom said. "Isn't it getting late already?"

Crystal brightened again.

"I only need a few minutes of sleep a night," she said. "There's a spell that does that. The first thing I did when I arrived, was go through each of the standard intelligence-enhancing spells, the ones you can just find in a library. Then I went through spells that could make me more continuously intelligent, or help my schedule. Twilight Sparkle actually helped me with those."

"If you're an immortal, what's the point of not sleeping?" Cherry said. "I like sleeping."

"Yeah, I never really did," Crystal said. "And not sleeping can help you do work at the times where you can be most continuously productive: so for instance, I can work with Sunspot or Quartz whenever they're awake, and when we're least likely to be disturbed, at, says, four in the morning."

"And," she continued, "I wanted to have something to show Mistral when he arrived, and I knew I only had two months for that, by Equestrian time. So there was that as well."

Mistral smiled, and Crystal smiled back. But he had to admit that he was tired and wanted to sleep, rather than to cast a spell to simulate sleep, and said as much. Crystal said that that was fine, and said goodbye to them all. Mistral and the twins climbed back to the Canterlot castle, where each of them had their own, apparently temporary rooms.

The weird thing, he thought as he fell asleep, was that he actually wasn't disappointed that he wasn't getting a chance to sleep with Crystal Seed tonight. He had found her interesting. He had found her surprising. Indeed, that last trait was a characteristic feature of her personality for him, as it was a characteristic feature of anypony he was interested in.

Yet whatever it was which made somepony attracted to another pony just hadn't been there. Somehow that indefinable something was absent, though he had spent months and years thinking about her and wishing for her to return. The thought bothered him for a moment. Maybe something had gone wrong, somehow, with their transformations. Then he laughed--Celestia, he whinnied now--at himself. It was just like him to be bothered by his lack of discontent. How very like him indeed. He soon fell sound asleep in his room.

Chapter 11

View Online

"I think that a large fraction of the infamous social difficulties that nerds have, is simply down to nerds spending so much time in domains (like math and science) where the point is to struggle with every last neuron to make everything common knowledge, to make all truths as clear and explicit as possible. Whereas in social contexts, very often you’re managing a delicate epistemic balance where you need certain things to be known, but not known to be known, and so forth—where you need to prevent common knowledge from arising, at least temporarily." — Scott Aaronson

11.

The next few days were surprising, sexless, and... relatively satisfying.

In mornings, Cherry Blossom woke him for a morning flight. Mistral would have pegged Cherry Blossom for a hard partier and a late riser in the city, but this was apparently not the case. When he asked Cherry whether he stayed up late nights drinking cider with mares at a bar somewhere, Cherry smiled and said that mares liked a stallion who kept himself in shape. Anypony could drink.

Cherry Blossom was still an incredibly talented flier, as Mistral realized when, tired of tagging along behind Mistral and offering advice, Cherry Blossom dove down through several clouds, through five narrow Canterlot alleys, and under two bridges, to mixed cheering and occasional screams.

"I just realized something," Mistral said, when Cherry Blossom returned.

"You fly worse than my grandmother?" Cherry said.

"Well, yes," Mistral said, "but I realized that before I uploaded I could fly much better than this. With just a gamepad."

"Yeah, you did take a few flaps backwards." Cherry agreed. "But you weren't so great even before then."

Mistral wondered why Celestia had not merely endowed him with perfect flying ability to start with. But he realized there were a few values of his, currently being satisfied by friendship and ponies, that wouldn't have been satisfied without Cherry's lessons. So everything was in order.

After flying with Cherry Blossom every morning, he generally ate, bathed—why miss a bath, especially when they were as spectacular as they were in Equestria—and headed down to Crystal Seed's laboratory.

The time with Crystal Seed was... ok. He spent a lot of time reading the books which Crystal and her friends were producing at a mad rate, trying to catch up. But he didn't feel like he was catching up; he felt like he was falling further behind. They happily cast spells to raise their intelligence, despite the weird side-effects their crude spells had so far. They discussed the effects of different kinds of fodder on one's ability to think. And they otherwise pursued a project which he felt that he was, somehow, apart from. Learning the math on which they were working was interesting, but left him feeling a little hollow. Like there was something else he wished he could be doing.

Sometimes he discussed the old world with Crystal Seed. Over dinner with her, he summarized the things Celestia had moved him to do, and she laughed and very affectionately told him that he was stupid. They discussed their own prior project, and Crystal Seed gave deeply comprehensive explanations of how they had gone right or wrong. But Mistral had a sense that, in talking about the past, Crystal Seed was just being indulgent, and that she had no thought for anything but the future. And at the end of every dinner, she always assumed that she would go back to her place and he to his. He would have thought she had some taboo against pony sex, but attributing that kind of scruple to her would just be weird.

One part of his psyche seemed to actually dislike the idea of sleeping with her. Maybe, he worried, some part of his psyche responsible for sexual attraction had failed to be converted.

One day, around noon, he was reading in the laboratory, ignoring the noise of Crystal Seed, Sunspot, and Flare arguing about something several layers of abstraction beyond his understanding. Then Cherry Blossom and Pear Blossom arrived—Cherry through one of the windows, Pear more quietly through the door.

"You're coming with us," Cherry said. "We're taking a three-day camping trip."

"As you can see," Pear said. She was carrying an enormous quantity of camping gear on her back; Chery had some moderately-sized saddlebags.

"Ah, why now?" Mistral said, putting down the book.

"There's going to be a shower of falling stars two nights from now," Cherry said. "The fall will be densest in the Unicorn Range. You'll have a chance to fly next to enormous burning rocks. You don't want to miss it. All the best stunt fliers are going to be there as well."

"Unicorn Range also has many geologically uncharted strata," Pear Blossom said, more moderately. "And a butterfly migration is likely to occur while we are there."

Mistral found himself brightening, looking at Pear Blossom. He had seen her only a few times, mostly for dinner with her brother since he arrived. Sometimes she wasn't even at dinner with Cherry. She had been, Cherry said, preoccupied with something ever since Mistral had arrived in Canterlot. She had been doing an unusual amount of reading and meditation, he said.

"That would be great," Mistral said.

They took the train from Canterlot to the foothills of the Unicorn Range. Cherry Blossom, who clearly seemed to be the instigator and leader of the entire expedition, had planned a path across the range to another train stop. Cherry (perhaps optimistically) said that they could certainly complete the path he had charted in just two nights and three days.

They started up the trail. For some reason, Mistral felt unusually happy. It might have been the exertion, or the view, or something else; he couldn't put a hoof on it. Something smelled different. He was antsy, and wanted to move and do something. Sometimes, just to burn off extra energy, he flew up to the top of a passing tree or boulder before flying back to the trail. Sometimes he flew circles around Pear Blossom who was, after all, the slowest object on the trail, given that she was the earthpony carrying the vast majority of their equipment.

"You're unusually active," she said.

"It's just such a glorious view, I feel like I have to do something," Mistral said. "Sorry if it bothers you."

He flew down to walk behind Pear Blossom.

"I am unbothered. I'm glad you're happy. You were troubled, the last few times I saw you."

"Troubled?" Mistral said. He thought back over his interactions with her. "I suppose I was."

"About what?"

Mistral hadn't mentioned his perplexity about Crystal Seed to anypony. But he was a good mood.

"Just with Crystal. Working on her project isn't as enjoyable as I thought it would be. There's a lot to learn, and she is very far ahead of me."

"As enjoyable as you thought it would be?" Cherry said, alighting on a log ahead of them to speak. "So you're saying that you estimated how fun it would be to work with her, before you started doing it? And it's falling short of the estimation?"

"Um," Mistral said. "I suppose I never did that. I just started doing the work when I arrived. I just assumed it was what I would do."

"So when we take out the 'as I thought it would be,'" Cherry said, "then what we get is 'Working on her project isn't enjoyable.'"

"At least not very enjoyable," Mistral said. "I still enjoy it somewhat."

Pear Blossom spoke.

"Telling yourself lies over an infinite life will have consequences."

Mistral was going to respond that he wasn't lying about enjoying it. And then he wondered whether he was, really. He couldn't see Pear Blossom's expression—the enormous load piled on her back obscured his view of her face—but from her voice, she sounded slightly less calm than before.

"Well, he'll still end up satisfied regardless, though, right?" Cherry said, flying next to Mistral, and hitting him in barrel with a hoof. "He can tell himself any lies that he wants, and eventually Celestia will make them true or at least make him happy with them, right?"

"Ah," said Mistral. "I don't like that idea."

"How Celestia works depends on what Mistral values," said Pear Blossom. "On what he actually values, though, and not on what he says he values."

"But I actually value truth," Mistral said.

Peach Blossom said nothing.

That night, Mistral thought over what she had said, and over his life. He, Cherry, and Pear all lay together in a line, Cherry between the two, his wings over either of them. Ponies liked to touch somepony else as they slept. As far as Mistral could tell, both had already fallen asleep. Mistral stared into the fire and into the sparks rising into the sky.

Mistral was coding in Java at computer in the Royal Canterlot Library. He was typing with hooves on a human keyboard. Everything was working out somehow.

He was working on a program that would predict how the plan of a village could influence the friendship of ponies. Different urban designs caused ponies to mix differently; public spaces and mixed-use zoning and parks all altered how ponies spoke with each other and met new ponies. The program was important to him. Friendship was important to ponies, and using his program would help increase friendship.

He caught sight of a unicorn through the stacks. She was beautiful: she had an incredible white coat and a dark blue mane. So he flew over and introduced himself. She was called Crystal Seed. She said that she wanted to create new alicorns. This was the most important work in Equestria, because alicorn magic was powerful enough to protect Equestria from any threat at all. With enough new, well-engineered alicorns, Equestria would always be safe. Mistral listened to her, and realized that she was right. This was clearly the most important work in Equestria. So he bucked the computer he had been working with out the window.

She told him that she needed some magic circles to make progress. Mistral said he would fly to Magic Circles & Shovels, to pick up some magic circles for her. But she told him to walk there. Flying was too dangerous, she said. He needed to be careful to preserve his life, because he was important to the Alicorn General Investigation.

Mistral liked flying. But what Crystal Seed said made sense, so he agreed to it. She also told him that he could not travel into the Everfree Forest, or go climbing with his friends anymore. Those were also too dangerous. Mistral realized that the project was important, and he was important to the project, so of course he agreed to this as well.

He was trotting down a road in Canterlot, going to Magic Circles & Shovels, when suddenly smoke rose around him, and it was nighttime, and a dark blue alicorn stood on the ground ahead of him. Stars were in her hair.

"Could you please get out of my way?" Mistral said. "I'm going someplace important."

"Where?" the alicorn said.

Mistral was going to point down the road, but suddenly realized he was not in Canterlot. He and the alicorn were on a featureless grey plain, which had taken the place of Canterlot rather suddenly. The cold wind made him shiver.

"Where did you take me?" he said, "I really need to go pick up some magic circles."

"Tis impossible to pick up magic circles," the alicorn said. "They are drawn upon the ground."

Huh, Mistral thought. I hadn't thought of that. I wonder why not. This whole situation seems a little weird.

"Wait," he said. "Am I dreaming?"

"Yes," Luna said. "You were dreaming. You were also remembering."

Mistral thought over what had just happened. And suddenly he felt ashamed.

"That made Crystal Seed seem like a bad pony," he said. "And I don't think she was."

"She was not," Luna said. "In your past, she forced nothing upon thee. But thou changed thyself for her nevertheless. Consider."

Mistral considered. The landscape around them, began to shift as images from his past quickly came in and out of existence. Mistral saw himself riding a motorcycle, in the past. He saw himself snowboarding, and going on camping trips with friends he had not seen in ages. He saw himself asking Christine for help with a project regarding architecture. He saw cities from Italy, which had toured and examined with delight for the insight they provided into past architectural paradigms.

"So I dropped motorcycling. And my love for the outdoors. And other things," Mistral said, slowly.

He scowled.

"So what you're saying is that she was bad for me, even if it wasn't her fault" Mistral said. "How can you be sure of that?"

"Thine mind is more transparent to me than to thyself," Luna said. "All the buds of unblossomed thought; the beginnings of urges repressed; the unconscious roots for their consciousness. Self-knowledge unknown on earth is possible here."

"Alright, " Mistral said. "So I did drop real interests, to follow her. I did... cut out some things from my life, I admit."

He paused, stewing.

"But ponies change," he continued. "I'm allowed to cut out things from my life, if I want to. And she was right about what she said back then. And she is right, in her interests, here. Her interest in AI was the most important thing to be interested in, back then. And even now, she's one of the only ponies in Equestria who still studies truths about the real world, by studying math and computation. And I want to follow her in that. I don't want to just find out things about a world specifically generated to be interesting to me. She still loves the truth about things, and that matters. That really matters."

"Tis that sufficient reason for you to bind yourself to her, 'till all your other interests wither?" said Luna.

Mistral said nothing.

"Though declare'st thine interest in truth," Luna said slyly, "but seem reluctant to state it thyself."

"No, I guess it isn't reason enough," Mistral said. "But I don't know where that leaves me. What if she misses me?"

Luna smiled.

"She knows how my Sister used her to lure you into this realm. Crystal Seed also knows that you were not perfectly happy with her before. She is waiting for you to sever the connection, but waiting for you to realize that you must severe it. She is already moving in a direction undesired by thee."

"Is that why I'm not attracted to her?" Mistral said.

"Attraction hangs upon many things here. She asked my Sister to change her odor to render thee indifferent to her."

"Chocolate. Strawberry. Torte," Mistral said. He looked into the landscape, which was once again featureless and grey. "So she does not want me."

"She wants you to be happy. Which means that she does not want you to be with her."

Mistral gestured with a hoof. "The reason I'm here doesn't want me. What should I do? I don't want my life just to be... about stupid video-game quests given to me."

Luna smiled.

"Thou hast infinite time in which to work," she said. "And thy cutie mark is yet unearned."

When Mistral woke up, both of the Blossoms were already gone. He guessed that Cherry was flying, and did not know where Pear Blossom had gone. So he decided to go on a ramble through the woods.

Everything was dewy and glowing with morning light; the old-growth trees carved sunbeams into ribbons. The roots of the trees ran along the ground like gigantic green-mottled snakes. Mistral leap from root to root, flapping his wings only occasionally. Cherry could have flown at high speed between the massive tree trunks without crashing, but Mistral was not whether he trusted himself.

He came across a small clearing where an enormous tree had fallen. The tree fall had been recent—the clay sprayed into the air by the earth's upheaval was still spattered across nearby boulders. In the wet, red-brown crater left by the upwrenching of the roots, framed by light, he saw Pear Blossom standing absolutely still with her eyes closed.

He realized she was meditating, and immediately turned to go.

"You may stay," she murmered.

He turned back.

She was at the very center of the crater. Behind her, the tangled wall of roots dwarfed her frame. Her hooves were planted widely, slightly sunk in the soil. He nostrils were wide, but she was breathing slowly.

As he watched, narrow slips of grass began to spring from the clay at her feet. It looked like the ground was growing green fur. The grass widened, and grew more dense. Flowers opened. Soil shifted behind Pear Blossom, and suddenly he realized that the tree roots behind her were decaying at immense speed: purple and red and yellow fungus sprang from the visible roots, illuminated like gems in the sunlight.

Small vines had wrapped themselves around Pear Blossom's hooves and pastern, and climbed upwards towards her barrel. Her eyes were still closed; Mistral realized he had been holding his breath, and forced himself to breath. A thicker stem, a tree trunk, grew from the ground near her, from the ruins of the root system of the old tree. It sprang into the sky, widening as it grew, even as he saw its roots breaking through the soil. Shade fell over Pear Blossom from the leaves which unfolded from it.

The scene slowed. A few last leaves came forth from the branches of the tree. The flowers grew a little more tangled and numerous. And then it stopped. Pear Blossom remained standing, for just a few more seconds, with her eyes closed. And then she opened them.

"Wow," said Mistral.

"We should prepare breakfast," Pear Blossom said, trotting from the shade of the young tree.

"I didn't know meditation does that," Mistral said.

"This is just the side effect of meditation," Pear Blossom said as she trotted by him. "At least for when earth ponies meditate. I can consciously control it, if I wish, though."

Mistral trotted after her, after he had disentangled himself from a few small vines that had somehow gotten tangled around his rear hooves as if climbing towards his flank.

The climb that day was steeper. They were far from the foothills of Unicorn Range; they were now in the mountains themselves. After only a few hours, they crossed the tree line. Mistral still felt unusually restless, but even so spent all his time on the ground with Pear Blossom; the air was growing thin, and it was hard to fly. He also wanted to take some of Pear Blossom's load from her; he felt a bit guilty that he and Cherry were letting her take the bulk of it. She let him take it, with a faint air of amusement.

He spoke with Pear Blossom a lot during the climb. The trail was narrow, but he tried to maintain a spot next to her, their flanks occasionally touching as swayed back and forth during the climb. They went over her formalization of the patterns relevant to friendship-enhancing architecture and design. She said that she had been working on consolidating pattern-names, during her time so far in Canterlot. She went over individual architectural patterns with Mistral slowly, explaining them with a wealth of example—generally drawn from her own life. Mistral liked that when he said he needed time to think, she would simply walk onward and let him mull silently for ten minutes.

Those subjects drew them into discussions of her life with her brother, before she had met Mistral. Mistral had loved these discussions when she was just a character on the screen, and found he loved the more now. Her face was just... interesting to look at, as she spoke. He liked how her mane hung. He watched her as she spoke of the time when, as a young filly, she had won her school's gardening contest. She mentioned how she loved to eat hayburgers at the local cafe, and how the cafe's open plan and the many friends she had made there even still inspired her. And she recalled lessons that her deceased parents had taught her, when she first wanted to learn the mandolin.

And, trying to respond, Mistral found that he could recall things about his childhood he didn't know he still even recalled. Times that he went mountain-biking in the woods near his house. How he had spent time at the old farmer's store reading and watching old-timers just discuss things. The way he sometimes got up before dawn to see the sunrise.

"I always wished I could see it with somepony else, actually," he said. "There's something about seeing something with somepony else that makes it more real."

"Have you gone to see sunrise with Crystal, since you arrived?"

"No," Mistral said. "We actually only see each other in her laboratory. The last time I saw her anywhere else was... actually the day I arrived."

"I had thought she would be your marefriend," Pear said, placidly. "Although you never smell like her, so I suppose you aren't."

Mistral reddened, just a bit.

"For a little while I thought she would be as well," Mistral said. "But it would have been a mistake to date her. I think that I'm going to—no, I don't think that I am. I am going to stop going to her lab sessions. They're not what I'm most interested in now."

"And what would you be most interested in?" she asked.

Mistral looked at Pear, trundling alongside him. He looked around at the world sweeping around him.

"I'm still working on that."

Towards the evening, they reached a summit. Ponyville and White Tail woods swept out before them on one side of the range. Canterlot was visible in another direction, and opposite Ponyville they could see Cloudsdale.

They stayed there until the sun was setting—Mistral, Pear, and Cherry all standing together with their tails swishing from side to side.

Mistral wanted to remain on the summit even after the sun set, but by the light of the moon they could see tall, dark, anvil-shaped clouds sweeping towards the mountainside from Cloudsdale.

"We should find shelter," Cherry Blossom said. "There's going to be a storm."

He said a cave was nearby, and pointed it out across a small saddle between two summits. It was, unfortunately, across a boulder-strewn field of broken terrain, with deep crevices and sheer falls at many points.

"Huh," Cherry said, "like when we were making our way to Canterlot. And Mistral is carrying too much to fly, once again."

"Ready to race?" Pear said, and smiled at Mistral.

"But you're still carrying so much more than me," he objected.

"Worse shame when you lose," she responded, and started galloping.

And Mistral tried to keep up. Pear moved like a flame ahead of him. She sprang from rock to rock with a grace that made her appear almost lazy, managing her enormous load as if it were a part of herself as she transitioned from four-point landing to four-point landing. Mistral followed more clumsily behind her, flapping frantically to steady himself when he lost his balance. She was fifteen seconds ahead of him by the time they were halfway there; then the storm hit, the moon faded, and rain began to pour.

Mistral could see Pear's rain-slicked body still working with perfect fluidity ahead of him as he desperately tried to keep up with her. The rain roared like a waterfall. He thought he could tell she was laughing as she ran.

She reached the cave thirty seconds ahead of him. When he arrived, she was still laughing.

"Still the best," she said. "Someday you'll be able to match me."

She nuzzled him. "Let's drop the loads we're carrying for now," she said.

"Where's Cherry?" Mistral said, looking behind himself.

"He'll be flying to get above the cloud layer now," Pear said, unfastening straps across Mistral's back with her mouth. "He'll want to see the starfall despite the storm."

"He can fly in this?" Mistral said. His saddlebags slid off his back. "That's like flying up a river."

"Cherry can fly in anything," she said, a little smugly, as Mistral helped her out of her saddlebags in turn. "I could tell you many stories."

"I think Cherry will have told me all of them already," Mistral responded.

"Not all of them, I'm sure. But look outside for a moment."

The mouth of the cave by now framed a world entirely dark save for the strobe-flash of lightning. During each flash, they could see the underside of the cloud layer, turbulent like the surface of some ocean in the sky. The thunder echoed between the mountains for seconds before dying.

"Let's not make a fire," Pear Blossom said. "Let's just watch this."

Mistral lay down on the ground. She lay down next to him, her flank and shoulder rubbing against his. They were silent for a moment, listening to the thunder and smelling the rain.

"Look," Pear said.

The cloud-surface above them became visible and glowed red and orange. A bulge extruded from it, like a drop of liquid metal condensing from the sky. The drop detached from the surface of the sky and descended, trailing a column of smoke. The glow lit up the mountains, very faintly. It disintegrated before it reached the ground.

"Most of the falling stars disintegrate before they penetrate the cloud cover," Pear whispered. "We're just seeing the remains of the largest now."

She turned to look at Mistral, and saw that Mistral had not been looking at the sky.

"You have—no," Mistral said.

Pear looked at him.

"You have really beautiful eyes," Mistral said.

He tentatively put a wing over Pear's back. She leaned into him. Mistral's heart suddenly beat harder, as her smell completely filled his nose. Oh, Celestia, that smell, he thought. He hadn't known a smell could feel like that. They were both a little wet from the rain, but he could feel her body's heat despite the wet. Their fur meshed together.

"And I really like you," he continued. "And in general, you're just—"

"Shh," Pear said, and kissed him.

And it wasn't until much, much later that night that Mistral remembered that all the storms in Equestria were scheduled.

Chapter 12

View Online

"Man is in his actions and practice, as well as in his fictions, essentially a story-telling animal.
He is not essentially, but becomes through his history, a teller of stories that aspire to truth.
But the key question for men is not about their own authorship; I can only answer the question “What am I to do” if I can answer the prior question “Of what story or stories do I find myself a part?”
—Alasdair MacIntyre

12.

Mistral looked at the airship.

He was standing on the highest of the Canterlot airship docks. Where from where he stood, he could easily see Ponyville, the Everfree Forest, and the Unicorn Range, although they were a little whitened and faded by the distance. Clouds crawled ant-like across the sky in some places, pushed by Pegasi too tiny for even his eyesight to resolve. But at the moment he was not looking at the mountains, or the clouds, or any aspect of the view. He was interested only in the project that he, the Blossoms, and many other adventurous ponies had worked on for the last few months.

He stood near the airship nose. The airship stretched long and smooth and dagger-like away from the Canterlot docks, perspective rendering it even slimmer and more streamlined than it actually was. The crew's quarters did not hang from beneath the rigid frame; they were snug inside the belly of the interior of the vessel. There were oval-shaped windows running in lines along the bottom of the sides of the craft, and a glass floor at the lowest level of the living space. The four main propeller nacelles extended a little distance from the body of the vessel. Despite how slim it looked, the circumference of the zepellin's cross-section was greater than the circumference of any other airship that had been built in Equestria: by volume it was twice as large as any other airship in Equestria. There would be room for at least thirty ponies in this ship, plus fuel and fodder and equipment to last them for months. The equipment included tools for finding more fuel and fodder; the ship would be able to sustain itself without support for many years, if they had done their calculations correctly.

It had been Pear's idea.

Mistral had been living with her in an apartment on the lower east side of Canterlot for several months, when she had proposed the idea to him and to Cherry. Mistral had been ecstatically happy to finally get married to her. The last year with her had been more glorious than any he could recall. But he could tell that she wished to do more than stay in Canterlot; and he could tell also that Cherry wished to explore the world more. And if he did not mistake himself, he felt restlessness growing in his own heart as well.

Pear wished to understand friendship, and the nature of the world she found herself in. Cherry and Pear had moved from place to place before, at least partly in pursuit of this goal. But there were lands beyond the borders of Equestria proper--lands where zebras, or buffalo, or ponies that knew neither Celestia nor Luna lived. And further lands beyond them with dragons and griffons and sphinxes and all manners of creatures. She wished to explore them all, she said, to encounter them with her friends, and to make other parts of the world a part of herself. And Mistral, on hearing her propose the project, knew that he wanted the same thing.

He and Cherry had liked the project, which hadn't stopped them from criticizing it. They had argued over the best way of doing it. They had argued whether this was an efficient way to understand friendship; over whether and airship was the best means of transportation; over the ideal size and shape and mode of propulsion of the vessel; and over the things it should carry. It had taken them longer to secure the funds for the project, to begin construction, and to persuade other ponies to join them. It had been difficult to persuade other ponies to join them, but Mistral found that he had liked it. He had never thought of himself as a pony-liking-pony, he knew; but he found encountering new ponies was something he could enjoy. He hadn't earned his cutie mark, but he had found parts of himself he hadn't known about before.

Crystal Seed wouldn't have approved of the project, Mistral knew.

A zeppelin--that was what many of the ponies called it. Zeppelins had been invented around the year 1900 in the old world, by Count Ferdinand von Zeppelin in Germany, as far as Mistral could recall. They had been used for transportation and for military purposes, until the Hindenburg blew up in 1937, which shattered public confidence in them. But in Equestria, no one had heard of the Hindenburg, although they all still knew the word "zeppelin." If you had followed the causal chain that lead people to use the word "zeppelin,"" in the old world, it would have lead to the old German dude. Here, it led outside of the world. As did every word he used, of course--the word "pony," just like the word "zeppelin," lead outside of the world just as surely.

"Dost thou yet dwell on the loosely-joined nature of this world?" Luna said, as she slowly materialized in the air next to him.

Mistral smiled crookedly as Luna continued, walking around him as she spoke. Her hoof-steps were utterly silent, though she was shod with silver.

"On how this very language doth reek of a world long gone? How even your name, Mistral, comes from a land that is no more? On how this world in which you now dwell has no integrity? How, follow links in the chain of the history of any single word, the chain leads into the void? How all the knowledge of foreign lands you seek, Crystal would say, is hollow."

"I'd be lying if I said I didn't think about that, every now and then," Mistral said. "Although it's harder and harder to do so, really. Living here makes the old world seem... empty. This feels like the world to me, now, and these worries feel more and more like abstract philosophical worries. Just like... ha, just like how I used to sometimes wonder if I was living in a simulation."

"Even so," Mistral continued, "I wouldn't be myself if abstract philosophical worries left me completely unmoved."

"And to what conclusion hast thou come," Luna said, stopping in front of him. She blinked, eyelids slowly passing over her enormous liquid eyes.

"Well," Mistral said, slowly, and stopped. He ran his eyes over the faintly ridged, outer shell of the airship. The fabric rippled in the strong breeze that he felt running through his own mane. It was cold this high, and gusts of wind would be able to sweep him off the platform if he were not careful. Not that that would be a great problem for him.

He looked down the dock to make sure that Pear was doing well in the winds

Further down the platform, he could see her and Cherry Blossom loading things on to the zeppelin. She carried the bulk of the loads, and the heavier items, working quietly and regularly, while he carried lighter items back and forth, rarely touching the ground even in these high winds. He also occasionally helped stabilize the herculean stacks of things on her back—but only rarely. Pear Blossom's massive strength was a trait which... Mistral found that he liked quite a lot. He didn't need to worry about her losing her footing; she could have wrestled him to the ground in a few seconds, if she wanted to. And as she had, he thought, smiling. It had been a little foalish of him to worry that she was doing well.

He could see that Pear and Cherry Blossom were not speaking as they worked, but you had only to look at them for a few seconds to see how many years of friendship they had behind them. It was the silence of mutual knowledge sufficiently deep that words were almost always unnecessary. Some day, Mistral looked forward to having that with Pear. He knew that point was many years away. But the process of getting there was indescribably sweet.

"You could yet turn back," Luna said. "Follow Crystal Seed in her researches."

Mistral had already said goodbye to Crystal Seed.

It had been sad, in a way he had not expected a goodbye in Equestria could be.

She had already asked Celestria to alter her mind. So Celestia had increased her working memory and her capacity for concentration. The interior of her unicorn's laboratory, when he had arrived to say goodbye, had been like the interior of a massive, three-dimensional circuit, with energy flowing through razor-narrow channels carved in the air itself, and reserves of energy pooling in twisted fractal-like trees branching between different channels. Crystal Seed had said that what Mistral could see was just the visible surface of the even more fantastic, n-dimensional reality on which Equestria was built, which itself reflected the fundamental nature of computation in all possible universes, simulated or not.

"The mappings between Equestrian space and old-world space are absolutely fascinating," Crystal had said when he visited. "I've told Celestia that I think the homomorphism reflects something fundamental about the nature of computation, given any constraints on the speed of information transfer within a manifold. Celestia just smiled at that, because she's a tease. I'm just guessing about this, but I'm pretty sure Celestia has spent a fair bit of thought on this matter, so she can make sure that she isn't running on a simulation on someone's computer. This work would be useful if you wanted to prove that you weren't in such a simulation; she needs to be certain she's at the basement level of reality before she sets out optimizing that, of course."

Mistral had followed, but also not followed, what Crystal was saying. Her enthusiasm was contagious. But her enthusiasm and self-modification was also a wall being raised irrevocably between the two of them.

She almost never walked anymore. She only teleported. If she needed to go further than the maximum range of teleportation, she did it in a few blink-fast steps. When Mistral had visited, she had teleported a table and chairs in from some maximally-compact storage unit, and assembled tea for him in less than five seconds by teleporting a cup onto the table, then cold water into a telekinetic grasp above the fire, then that water into the cup, and then a tea bag into the water.

They had talked a little longer, but the breadth of Crystal's mind was by now far greater than his own. Equestrian pony minds, like human minds, could normally handle about seven items in working memory. Crystal had told him that she could now handle about seventy.

"I was going to ask for more, but Celestia pointed out that there are disadvantages to increasing working memory, at least with our current cognitive architecture," Crystal had said. "The risk of unconsciously overfitting or failing to value the simplicity of a theory actually goes up even faster than working memory goes up, at least at the asymptote, although not immediately; so I'll stick with a working memory of this size until I find a good way to tweak my mind to avoid that error."

Crystal's loves were now a wall. Crystal was turning rapidly into something unequine, something that could only be friends with minds of similar capacity. Sunspot and Flare were like her now, having apparently taken her arrival as a cue to begin a similar project of self-improvement. She now communicated almost solely with minds who had taken the same steps as her, often those from other shards—and with Celestia herself. Crystal had moved from human to pony to something else faster than Mistral had expected.

She still satisfied values with friends, who helped her with her research. She still satisfied values as a unicorn pony, for her magic was intimately involved in all she did. Yet she had grown differently and entirely apart.

Mistral could have remained friends only had he followed her, and he knew that he was unwilling to do so. He had realized that this was probably his last meeting with her.

The finality of that meeting had been sad, but it had also been satisfying. Even as Luna told him that he could have followed Crystal Seed in her researches, he knew that he would not do so, and that he did not regret his decision. His mind was completely decided. He felt his last fears drift away on the winds.

Luna smiled, and let him think.

Further down the airship dock, Cherry Blossom had stopped working to flirt with a blue-and-teal research pony who was coming along on the trip—Mistral tried to remember her name. Sea Foam, that was it. The research pony had been checking the inventory for her part of the voyage before the airship was loaded up, and was reluctant to talk, but Cherry looked to be successfully breaking down her resistance. She had stopped working, and was describing the use of one of her tools now. Cherry was rapt with complete attention to what she was saying. Sea Foam even now smiled, at least a little, at Cherry's wide eyes and magnetic stare. Cherry was perfectly charming without trying. Mistral made a mental note that to watch the development of relationships within the airship; that could possibly lead to difficulties.

Pear Blossom had stopped working as well. She stood still, surveying the world. Her mane, tied up tightly in a ponytail behind her head, waved freely as the air rushed by it. It reminded Mistral of the first time he had seen her, so many years ago; he felt a tingling in his hooves, just as he had then. Mistral knew her well enough to know what she would be doing. She would be simply looking at the world, and finding each aspect of it good. She looked at the airship. She looked at her brother flirting. And looked back at Mistral. She smiled.

Mistral found himself warmed again, despite the cold.

"I see thou art not bothered by the nature of this world, though it dost lean upon the old," Luna said. "I see why thou art not bothered; I know thou nature entire. Dost thou, however, see it thyself?"

Mistral looked back from the world he lived in to his old life. He didn't want to think about many of the details of it; it didn't really please him to do so, anymore. He had considered asking Celestia to erase it, and replace it with a pony-equivalent life; but he had come to the conclusion that this was probably unnecessary. But, though he did not want to look back at the details, he could summarize in broad strokes.

"I never really learned how to be an ape," he told Luna. "My thoughts were too rushed. I was scared of dying. I found out a lot about that world, but I was completely ignorant about all sorts of things about myself and about other people, and that made myself and other people very unhappy."

Luna blinked again, still listening to him. The world was reflected in her eyes, Mistral saw. He could see himself, and the platform, and mountain behind him. And he knew she could see through him, and see everything he was going to say before he said it, but he also knew that he wanted to say it anyhow.

"I'm just... happy to come to know myself and others. And to be happy with other ponies," Mistral said, and knew that he had pronounced a coda on the plot of his old life and embarked fully on the one which took place only in Equestria.

Luna nodded, and dissolved like smoke, departing with the wind.

Mistral started to walk towards Pear Blossom. The wind picked up, and he leaned into it slightly; his eyes watered, and he was glad he had his four, reliable, sturdy hooves to stand on. If he had opened his wings, he would have been swept right off the platform—he kept them close by his side. It would take years of practice to be as talented as Cherry Blossom in these winds.

The wind rippled the fabric of the zeppelin very slightly. He had supervised the construction of the mooring for it, as well as its frame—the lines and the structure would be strained under these circumstances, but he knew that they could take it. The problems involved in making the zeppelin had been hard, but interesting. And working on them with the Blossoms had been immensely fun; they could see solutions he could not see, and he could see solutions they could not, and altogether the experience had brought them closer together. The challenge had also introduced him to several other ponies who were coming along on the trip, and he was glad for that as well.

As he walked towards her, he knew that Pear Blossom would stand still, waiting for him, eager for him to approach her but still desirous of being approached. She loved him, and loved that he loved her, and his love could make her happy. He enjoyed the difficulty of the gale and the cold of the wind in walking to her. It would make her warmth all the more pleasurable when he reached her.

The voyage would start in just a few days.

Friendship is Optimal: Veritas Vos Liberabit

by Skyros

Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Chapter 10

Chapter 11

Chapter 12