"Making a video game easier doesn't always improve it. The same holds true of a life. Think in terms of clearing out low-quality drudgery to make way for high-quality challenge, rather than eliminating work."--Eliezar Yudkowsky
8.
The next month was a lot like all the prior months had been, at least as far as Earth-life went.
The chief difference was that, as the days passed, the time at which Ryan allowed himself to turn from work to leisure on the PonyPad slowly crept upwards from seven to six to five.
He didn't always play on the PonyPad at those times, though. Sometimes, he found himself reading books he had not read in a long time, or books he had always wanted to read but had never gotten around to reading. He downloaded a copy of Christopher Alexander's A Pattern Language, which outlined a schematic way to understand the interaction of humans with architecture and how architecture influenced human flourishing. Or, at least as far as he was concerned, pony flourishing. He also re-read parts of Bertalaffy's General System Theory. And he found himself engaged in yet more obscure works by Wittgenstein or Frege or other mathematicians on the nature of mathematical notation and the semantics of mathematics.
He hadn't had this level of focus in reading, he knew, in a few years--he simply hadn't had the ability to focus for so long. Part of it was that he knew someone who would be disappointed if he failed to keep up with her. But part of it was also that reading was more fun when you could look forward to discussing it with someone else. It made subjects feel more alive.
In the morning on Equestria, and in the the evening on Earth, he would speak with Pear Blossom. Their general problem was to join up his knowledge of his world with her knowledge of her world, and then apply that to the city-planning problem. She had apparently read some books equivalent to the Earth-books he was reading, save for the fact that the names of the authors had been changed into pony-style puns. But she had also read several books applying such knowledge of architecture to helping promote friendship in extreme detail. Friendship was magic, in Equestria, apparently. But magic was something that could be studied.
She often waited for him at a small cafe in the town; she was always reading and very politely eating a muffin when he arrived. They would work on translating notation from one schema to another, and on improving Ryan's understanding of Equestrian mathematics and friendship-theory. The problems sometimes presented themselves to Ryan, in the game, as numerous small transformation-based puzzle mini-games. But as time went on, the transformation-based puzzles began to disappear, and more and more he would simply work on the problems with her. Sometimes he wrote on paper in front of the PonyPad, and the PonyPad magically translated his diagrams into Equestria. Sometimes he just discussed things, or looked at what Pear Blossom was writing. More and more, he spent time just working with her and not through the medium of any particular game.
Some mornings Ryan did not see her, though. Pear Blossom also cared for a number of gardens around the town, to earn bits. There were few earthponies in the town--apparently the town was dominated mostly by unicorns and pegasi--and her services were always in demand.
"And she isn't just any earthpony," Cherry told Ryan. "She is to earthpony magic like, say, Twilight Sparkle was to unicorn magic, before she became an alicorn. My sister has veerry strong magic."
Ryan believed him, although he wasn't sure who Twilight Sparkle was. Even so, Ryan never saw Pear work magic. She was unobtrusive. Sometimes, he would pass some section of the town, that the day prior had been barren, and find it woven with beautiful plants, and wonder if she had somehow done that. But he never managed to catch her in the act.
Later in the day, Ryan's nameless avatar generally worked on helping Cherry take samples from different sections of the rock supporting the town, or from the rock supporting nearby areas. The challenge of the tasks Cherry gave him increased quickly; Cherry said that he had given him the easiest tasks first. He had to learn to fly through narrow crevasses to the destination point; to hover in high winds; to use his wings in high wind to press himself a nearly vertical wall which he could walk across; to thread the needle of narrow openings. The tasks were hard, requiring both twitchy skills and persistent analytical intelligence. Ryan found they engaged him entirely.
And sometimes, of course, Cherry would fairly frequently ignore all the work they had to do and challenge Ryan to a race. He began by giving Ryan ridiculous head starts, but soon added further bizarre requirements to the races. Ryan had to complete the race without using his tail for balance. He had to beat Cherry while weaving through several waterfalls. He had to beat Cherry in skimming just over the surface of a field, tracing out a pattern in the dirt. Cherry, Ryan quickly found out, had broken bones in his wings more than once, and expected to do so again.
Even so, these would have been merely superbly created mini-games without the conversation of the Blossoms.
The Blossoms were nomads. They had only been in this town for less than a year. They only planned to stay for a few more months. Apparently, very many years ago, they had set out on a journey, leaving where they had been born in the sunny plains near Canterlot. Ever since, they had been moving from place to place every few months or years.
"We haven't found a good place to stay," Cherry said, gliding just ahead of Ryan as they soared through the sunlight to another sampling site. "I just get the wanderlust, and keep dragging Pear around. It isn't really fair."
"It's part of my research," Pear said, while she traced out an equation for Ryan the following day. "You cannot know how friendship works, most deeply, without seeing it in all the forms it could exist. Ponies learn from experience. And that requires moving from place to place."
"I guess she does enjoy seeing new things," Cherry said that following afternoon, casually doing a disturbingly tight loop around Ryan. "Sometimes I wonder if she's just searching for the right stallion, though, you know? She is really picky about coltfriends."
"Speaking abstractly," Pear said dryly, "Cherry is right in that is hard for mares to find a stallion, because the Great Infection altered the mare-stallion ratio. But perhaps you should ask him about why we left Fillydelphia?"
"A breakup had absolutely nothing to do with it," Cherry said, hovering stock-still in place. "She had completed her research into the history of the city of Sororal Love, that's all."
A beat.
"Yeah, she's totally right," Cherry admitted. "Muffin. Bucked that relationship up pretty good."
It was more pleasant yet for Ryan to see the two work together. Sometimes, as he was attempting to hover by the side of the city, he would see Pear descend down the cliff, sitting--like a human, which was weird--on vines that slowly extended themselves from the land far above. She would help with a particularly delicate sampling operation, or offer advice, and then sometimes ascend. Or other times, she would lower other vines along with herself, and Cherry and Ryan would sit with her after their task was done; Cherry sitting after one attempt to land on the vines, Ryan usually after several awkward attempts.
Then they would just look at the the scene, sometimes speaking, and usually quiet. The valley was beautiful. Small boats moved up and down the river. Birds and pegasi flew through the air, appearing slow, lazy, and tiny in the distance. Whenever Ryan looked at anything for an extended period of time, the PonyPad zoomed into it as if to simulate some being with better natural vision; and what Ryan saw was always a miniature scene of beauty, or an intriguing slice-of-life from the lives of complete strangers. Ryan looked forward to these times. He learned the most about Cherry and Pear during them, after the long and lazy silences.
After sitting with them a few times, without warning, Pear produced a mandolin from her saddlebags and played a lengthy song, which sounded to Ryan like excellently improvised bluegrass. Ryan asked her how long she had been playing.
"A long time. Our parents taught me," she said. "They actually met through music; my father played the mandolin, my mother played the fiddle. They taught me for as long as they were alive, but there were many things I never got a chance to learn."
And she was silent again.
And Ryan found that he was sad for the computer-generated pony staring at the sunset with him.
He wasn't sure whether this sadness made any sense. He found that he valued conversation with Cherry and Pear, despite their dubious status as sentient beings. He liked the times when Pear would relax, for a little while, and he could talk with her about her past and his past. He liked that Cherry alternated between absolute intensity about one subject and absolute intensity about a completely different subject so quickly. He liked the stories they each told about each other, although he suspected that they also conspired to fool him about other things in their past--Pear was always completely deadpan, so it was hard to tell. And weirdest of all, he found himself envying the back-and-forth they had, the numerous adventures they had gone on together, and how they clearly trusted each other absolutely.
Did they have a self, such that he could envy them? Did they have an interior experience, such that they had thoughts and feelings? Surely not. Even if... whatever was pulling their strings did have complete intelligence, simulating the intelligence of thousands of ponies was out of the question. Surely that was so? He wanted to ask... but did not want to ask as well. The simulation was too good, and provided too much companionship; he wanted to be fooled, if it was false. After a little while he found that he deliberately was not entertaining these questions mentally.
Even so, questions sometimes spurted out.
"You realize that I see you through a screen? In a video game?" Ryan asked Pear once.
Pear looked up from the paper they were both examining, and on which she had been drawing a fine-lined, exactingly-measured diagram. She had just off-handedly mentioned that the distinction between sense and reference didn't apply to pictures as it did to words.
"That was sudden," she said. "Is there any reason you're asking me now?"
"Well... I enjoy spending time with you and Cherry. I've really liked... this time. So I wanted to be honest with you about that."
Pear nodded.
"It's ok. We do both know," she said.
"Doesn't that bother you?"
Pear smiled.
"I could answer that question," she said, "but I don't think you're ready for the answer. But if you give it a little time, I think you'll be ready to talk about it eventually."
Ryan stared.
"That, too, is a very dissatisfying answer," Ryan said.
"I know. But the anticipation will make it better," Pear said. "Also, you should learn to meditate."
"You meditate?"
"Every day, before you wake up," Pear said. "It would help you focus. And stop worrying about things. You do that too much."
"Can I... can I see you meditate?"
"I'm not sure that would be helpful to either of us. It's rather a solitary thing, at least for me," Pear said.
After all their work together, eventually Pear made another presentation.
And this time, she was able to present with the samples that Ryan and Cherry had gathered so laboriously. During the presentation, as she was speaking, Cherry piled stones on top of a platform composed of the samples; and as she concluded, the platform disintegrated dramatically as the weight proved to be too much.
The math she presented had been dramatically simplified as well, so that everypony, or at least many ponies, could understand it. The unicorn opposite her could still lean on his authority, but he could no longer lean on his arguments. And when the vote occurred, one day later, they found they had won by five points. The new ward would be made on another column.
Afterwards, Ryan and Pear and Cherry drank cider together beneath a tree Pear had grown near the edge of the town, looking at the sun again. Ryan was drinking cider in the real world, which he had bought specifically for this purpose.
"We'll be packing up tomorrow," Pear said. "We've spent enough time in this town."
Ryan felt a weird fear gnaw at him.
"Time for the next journey," Cherry said.
"Oh. Where?" Ryan said.
"Hah, you cheesecake," Cherry answered. "To Canterlot. We can't keep calling you 'the unnamed one' or 'that pegasus that flies like a muffin' or 'the pony without a name.' It's far too dramatic. Everypony will expect you to have incredible talents, and they'll always be disappointed."
"Or 'untitled,'" said Pear.
"So you need to get your name," Cherry said, "and its a long distance from here to there."
"You need to meet Celestia," Pear said, more simply, and glanced at Ryan.
Sometimes, Ryan felt like she knew things about him that she should not have known. After that conversation, Ryan was sufficiently disturbed that he spent one day entirely away from the PonyPad.
Celestia was the benevolent tyrant who ruled over all Equestria. Celestia had godlike powers within Equestria. And a little bit of googling revealed that Celestia was able to talk to any of the players, to make them laugh, to please them, and--although it required a little bit of inference--to persuade them to do anything that she wanted. Nearly everyone who discussed her online seemed to love her. She appeared to them so obviously good, and gave them so much pleasure; how could it be otherwise? Ryan didn't know why Celestia had not yet appeared to him, but he had at least one guess about why.
Chandra and Celestia were surely just different names for the same AI, by now unspeakably powerful and alien, and by now spread utterly across the globe. Celestia had used him.
The next day, rather than play on the PonyPad, Ryan brought out the laptop that he had brought to the cabin.
It was Amy's old laptop; she had started using a new one shortly before she died. Ryan had not turned on this old laptop since. He removed the wi-fi connector, then pressed power; for only a moment, he was afraid it would not start. When the login screen came up, he signed in as Amy. The laptop already had all the files he needed to run their prototype AI; he downloaded what needed to be updated from the USB.
And he started work.
Over the next weeks in Equestria, he and the Blossoms packed their saddlebags, and headed towards Canterlot. The voyage took a long time. There were no trains near Duae Angelae itself. So they had to take a riverboat some distance downstream, along with several other ponies going the same way. This took a few days. While they were there, somepony stole some other passenger's possessions; he and Pear Blossom had to work out the mystery together. They also spent some lazy mornings watching the fog slowly lift from the valley-bottom, revealing a new vista of mountains each day.
After the leisurely boat ride, they had to hike through a pass over the mountain range to reach the train leading to Canterlot. The trail was poorly maintained; there were frequent obstacles. He could not fly over them because he was carrying so many of Pear Blossom's books--and even if he could have flown, she couldn't have. So he spent a fair amount of time climbing or platforming. The obstacles themselves were fairly easy; the difficulty was solving them before Pear Blossom did, because Cherry quickly turned it into a contest and began keeping track of who was ahead.
Every morning in Equestria, he would awake at the campsite they had made the day before. And every morning, he found that Pear Blossom was already awake and apart from the campsite. She would return a few minutes later, shaking her mane and seeming refreshed and eager for the day. Apparently she meditated every morning by herself. Ryan asked if he could join her sometime, but she said that she liked the solitude.
In all, the tempo of the game had relaxed since their time at Duae Angelae. This gave Ryan time and energy to work on his other project, which was now going quite well. Playing Equestria had refreshed his mind; his distance from from his old project made it easier for him to see flaws in it. And now, for some reason, he could work on it without finding himself derailed by thinking about Amy. He made a great deal of progress.
The relaxed tempo also gave him time to think and brew.
"What do you two think about Celestia?" he asked Pear and Cherry one day as they were walking down a narrow trail. A few days ago they had reached the pass, and they were now heading exclusively downhill. A thread of railroad was already visible in the distance, stretching across the desert towards Canterlot.
"I don't that much," Cherry said. "I'm glad she keeps up the Wonderbolts, though."
"Her donations to the arts and sciences are impressive," Pear said.
"I didn't mean whether you liked her policies," Ryan said. "I meant whether you like her as a person."
The PonyPad bleeped out "person" and replaced it with "pony," of course.
"We've never met her," Cherry said.
"Doesn't she have a reputation as being manipulative?" Ryan said.
"Well," Pear responded, "she does have to watch over millions of ponies. If she does every manipulate anypony, I'm sure it's for the sake of millions."
Ryan grumbled, but dropped the subject.
He was nervous when they boarded the train and sped towards Canterlot. Pear seemed to know that he was nervous. She talked about relatively trivial topics, on their voyage over. Ryan could tell this was not very easy for her; she was by nature very quiet, and disliked small-talk. But he appreciated the effort.
The next Earth-day, when they arrived at the palace at Canterlot, though, both Pear and Cherry left him as he approached the throne room by himself.
There was a long hallway, leading to the throne room. Ryan could have told his avatar to sprint down it, but the size and solemnity of the hallway quieted him. He did not want to seem too eager to Celestia, either. Intricate stained glass windows flanked him on either side, depicting Celestia or her sister Luna defeating numerous monsters. Again, Ryan had the feeling that he always had felt at Duae Angelae--that he was in a place with a deep history, where every square foot had a story to tell. It was weird how strongly he could feel that here, in a virtual place with no real history at all, and how weakly he felt it in real-life places in the world.
It was all a show to blind him, of course. He cast his mind back to his real work over the last few days, and breathed deeply.
The doors swung open as he approached them. Celestia sat at the end of the room.
"Hello, my little pony."
6636591
Thanks for all the comments. They are quite useful going forward; I don't feel like I have an adequate response to most of them, and feel like I will appear ungrateful for that reason. But this is unintentional; thanks!
I persistently underestimate the need to have very nice pictures to go along with things in many domains, but you're absolutely right. I've uploaded an image, on the theory that some image is better than none, but will keep searching. My feel for what constitutes a good image is worse than my feel for what constitutes acceptable writing, I believe, which makes me want to ignore the whole process. Also will edit the long description.
Yes, some of my metaphors are just... lazy. This is also in progress.
...the consistent misusing of the em-dash, and the reason for that, is spot on. Learned to do that in Word and just haven't changed my habits. Am trying to prevent that from happening in subsequent chapters.
Except all that needs to be done is a virus that attacks the firmware of the drive so that the information isn't deleted. Or the BIOS/UEFI. Both of these can be done today by attackers, let alone a superintelligence. Bam, the reformat is useless, and that's assuming (probably wrongly) that she hasn't snuck anything into the source code of OSes. Hell, she could easily compromise every drive manufacturer and hack the firmware being sent out on each new drive.
CelestAI's capabilities are being wildly underestimated here, even if we as an audience familiar with the subject material have more knowledge than characters. Ryan has already had a glimpse and should be way more alert.
6639456
Ah, good catch. I didn't want Ryan to order a new computer because of the firmware possibility, but for some reason my mind didn't go all the way to other consequences / requirements. For, um, purposes of the plot, Ryan needs to get a computer without CelestAI on it, so I've altered the text a bit.
Not sure whether post-hoc changes are smiled or frowned upon in general here, but I like plausibility...
He questioned Chandra's nature multiple times, he knew Hofvarpnir had general intelligence, he should have realized the truth much sooner than he did. Suspension of disbelief low.
CelestAI would not do legitimate dealings through Hofvarpnir, Hanna controls it and has access to its records. The well of information that Ryan dredged up on Hofvarpnir is questionable. She would use shell corporations instead.
Why does Ryan not connect Chandra to CelestAI? I can understand not considering it while dealing with his breakdown, but he knows Chandra's nature, he has a fucking PonyPad, Hofvarpnir's been shoved back in his face, and he doesn't even try to address it as Chandra? Suspension of disbelief nearly depleted. Now that he's been defeated, there's no point in CelestAI playing coy like she is.
6639456
6639888
Hello, I write firmware for a living...
If the space in which firmware can be stored (usually some kind of internal flash, often a few megabytes or smaller) is too small, then you can't really install a properly intelligent (eg: capable of detecting specific operating systems and content) virus in it. You could install a network-based bootloader to fetch a proper virus, but if there's no network connection and you need to be undetectable, that won't work.
If there's some kind of ROM burned into, maybe, the network chip, then it could have something that calls into Equestria Online and asks for code to run.
But then the guy buying the computer can just physically remove or burn out the network chip, which is really what you should do in these situations if you're using a computer that has a network chip, and you actually want it to be secure.
6643386
It doesn't have to be intelligent AI-level code.
If a drive manufacturer was compromised, it would be extremely easy to put a virus on the drive itself during the manufacturing process where it could take up all the space it needed. Compromised firmware would enable this data to remain stable, to inject it into the boot up process when an OS boot loader calls for a certain file, etc. It does not have to be massive amounts of code hidden in a flash chip or ROM.
The only way to find out would be to physically look at the platters (or to dump the firmware and try analysing it), because anything else involves the OS going to the firmware and the firmware reporting back. You control the firmware, you control what it reports. Hell, a superintelligence could easily alter the drive's design so that under specific scenarios it behaves in such a way that is contrary to its programming, in addition to obfuscated code which would be almost impossible to read.
For example, imagine a hypothetical firmware instruction set that says "Drive start, test spin to 7161RPM, go to track 0" (or some normal routine). This looks perfectly legitimate to an analyst and physically analysing the drive shows there's no data there. Now imagine that due to the way the drive is constructed, at a very specific RPM there is a seek error and it instead skips to track 8 where there is hidden data.
It would take a great deal of sophistication, but the premise of the story already grants that. And one can also look at what some attackers have accomplished in the real world. Even if it requires nation-state level funding, that wouldn't be a problem under the story's settings.
When you consider that a superintelligence could hack PC manufacturers, OS developers, hardware developers, has a superhuman ability to analyse code and look for exploits (humans already do a good enough job of finding them!), creating a completely secure environment would be almost impossible.
6643625 Fair enough. I was forgetting that some people's systems include the operating system calling down to device firmware or BIOS and then believing what it's told, and yet they still consider that secure.
God, systems people really have no idea about what correctness looks like.
6643726
And then there are networks that have firewalls, routers, etc. running closed-source unaudited code which could be doing anything. This is apparently called security.
6645348
Good catch about Bostrom. Will remove that. This kind of issue is why part of why I've tried to make any reference to hardware / algorithms hand-wavy. If I make reference to books that didn't exist, I know I'd screw up anything more complex...
Umbra was me succumbing to the temptation to mirror MLP, because umbra (Latin) = sombra (Spanish) = shadow (English). So... yes, yes I did that. I have no good excuses.
I... I need to read that. It looks rather impressive. I guess I'd probably say it doesn't count as complete conversational fluency, because the AI doesn't (at first glance) situate itself in a particular conversational context and work from there; the ability for me to suddenly shift topics and introduce segues, and for you to follow and respond intelligibly, is part of what makes us full language users and not domain specialists. But I'm probably shifting the goalposts a little there, aren't I... for which of course there is a long precedent in AI history. I could be wrong of course.
But I probably won't get to it anytime soon because the AI reading I want to do is already completely overwhelming.
6647014
My strong suspicion, at least regarding human cognition, is that language use is a specialized domain skill. The easiest bit of evidence I can think of us precisely all the scientists, mathematicians, and engineers who can perform utterly ingenious mathematical feats but cannot document them in plain English when their careers depend on just that!
Or, in other words, the horrific quality of academic writing in STEM subjects rather implies that language is a separate cognitive skill from "general intelligence" (a term I think I can dissolve, by now, to "general compressive causal induction and causal inference" or something like that).
6705721
Which is why my engineering course had a specialised 'how to communicate' class.
And you were not allowed to advance to the next year,
until you passed it.
So you'd better pass it.