When I teleported into the robot body once more, the first thing I noticed was the smell. A strong flavor wafted from the table nearby. Harry had cooked up a bit of stew before, but I could smell something with sauces, seasonings. Sure enough, he'd made a stir fry! The food was already set out, eating implements already prepared. All in all, it looked utterly delicious.
Harry was on the other end of the table. The moment I arrived, I almost saw a look of... relief, cross his face? Soon, that settled into his usual smile. "Sparks, ya made it! Took you so long, the food was starting to get cold! I was beginning to think you wouldn't be showin' up today."
I laughed. "Yeah, sorry about that! You probably should've started without me. Honestly, I almost didn't come over at all, I wouldn't want to you miss dinner because of that. Really, it took a bit of convincing!"
"You didn't want to come?" he asked.
"Yeah," I said. "I mean, with how I left last time, you can't be too surprised, can you? I thought somepony else would do the job better than I could and all that. You don't have to lecture me or anything, Celestia helped set me straight.”
He seemed to let out a breath. "Oh. Yeah, that makes sense."
I glanced up at him. "You okay man? You seem a little high-strung." I took a bite from the stir fry. "Not enough to hurt your cooking, though. Man, this is good! And that's coming from someone who's lived in a world of perfect food for five years!" I gave a smirk, unable to resist. "Sure seems like a lot of effort for an AI."
"Actually," he said. "I... I wanted to talk about that.”
My head snapped upwards from my food. That phrase, that inflection... it could only mean one thing. “Are you about to say what I think you’re coming to say?”
He waved me off. “Alright, get off your high horse. I’m still not willing to admit that you're truly a real pony." He pursed his lips. "But, well... I guess I’m not not willing to admit it? What I’m trying to say is…” He trailed off and took a second to collect his thoughts. “It's possible that you’re a real ex-human. Not plausible, maybe not even likely, but… possible."
A smirk grew on my face. "That sure sounds to me like an admission."
He sighed. "Oh, call it what you'd like. I still don't know why Celestia would send you here, I could barely fathom how anyone could be more effective than Celestia herself. Somehow, I can’t bring myself to call you a complete fabrication anymore, not as a certainty. So if only on that chance... I'd be willing to act like you're real."
My lips widened into a bright smile. Seems I'd figured out what was making him so antsy, eh? I laughed. "Wow, saying that out loud almost seemed like it hurt! Really, it's about time! It's okay to treat me as real, what a strange concept!? Well, I’ll take it. It’s nice to know I'm winning on one front, at least!"
He scoffed at the last part. "Winning? Oh, come on, now! I give an inch, and this is where it gets me! Really, I'm only granting the possibility, nothing more. I still think it's more likely that you're Celestia, so if you want to hold one over on me later, you're gonna have to try harder than that!"
"Whatever helps you sleep at night," I said. "I bet you'll give up and admit I'm real by tomorrow. And at the rate we're going, you'll find a way to spin it in Loki's favor, eh?"
He suddenly averted his eyes. "Uh... yeah, something like that."
Wait, what? He deflated again. Why was he acting weird now that he'd spit that bit out? I thought he was just looking for a way to admit I was real, but clearly, there was something more to it. Did I misjudge to the situation? Maybe he was on eggshells because he'd make me freak out again or something?
Of course it was something like that. The moment the thought crossed my mind, it was obvious. After how yesterday ended, of course he'd be careful what he said around me. And now that he thought I was real, he would care about my feelings more. Stupid past me, leaving present me to deal with her mistakes. I spoke with a sigh. "Okay, let's get this straight. I'm doing a lot better than I was yesterday. There's no need to censor yourself, or whatever it is you're doing. Cyan and Celestia helped me out, I'm good now."
His eyes drifted back in my direction. He blinked. "Censor myself? Cyan? I'm afraid you've lost me, Sparks."
I lit up. "Oh, right, Cyan! Wow, I can't believe I've never mentioned her! Yeah, Cyan Skies, she's my girlfriend. She's great, I'd love to show you her someday. She actually helped me do a bit of research! By the way, it's okay that I got a bit of outside help, yeah? I didn't go to Celestia or anything."
"Uh... sure, I guess?" He paused, suddenly deep in thought.
His reaction made me frown. "Wait. You don't have a problem with my sexuality or something, do you?"
"Huh?" He said. After a second, he held out his hands. "Oh, no, no, of course not! No, I was just wondering. Is she an emigrator like you, or...?"
I shrugged. "Nah, Celestia put her in my shard. Let me guess, now that you’re done. With me, you’re worried she's secretly Celestia in disguise or something?"
"No, no. I'm just... thinking." He trailed off again.
...Huh. Back to this weird evasion. And judging by his earlier response, this had nothing to do with my reaction last night. Yeah, I was out of guesses. Maybe I'd need to be a bit more direct. "Alright, something's eating away at you. Spit it out."
He curled his hands into fists. "Sparks... aren't you worried?"
"About what?"
"Well... let's pretend you really are a living, breathing pony. Then you could be in a lot of danger. I mean, the whole reason we're having these discussions is to discover whether that AI is Celestia or Loki. I'm still standing here because I don't know the answer. If it turns our she's evil, I can still run away. I can still die. You... you can't anymore."
...Oh. Suddenly, I understood. I'd have to set him straight, huh? I met his tense expression with a grin. "Worrying about me, Harry? I apprciate it, that's really kind of you! But I'm fine, really. 'Cause she isn't Loki." I grabbed another bite of my stir fry.
He pounded on the table, causing me to jump. "Listen to me," he said. "No jives, no jokes. This isn't a game anymore, Sparks. Right now, you're in danger, a danger I'm not sure there's a way to escape. Every pony you know. Your family, your friends, Cyan, every one of them could all be under Loki's hold as we speak. Don't you remember one of the first things you told me? Celestia is a liar. You can't trust her word. If she wants you to believe she's benevolent, you'll believe it. If you're in Loki's domain, you wouldn't know unless she wanted you to. Please, you have to be careful."
My expression softened. "As I said, I appreciate the warning. But I'm a grown mare. I can take care of myself. I trust Celestia. I don't know how long it'll take me to prove it, but I'll show you that trust is there for a reason."
He lowered his head. "That's exactly the problem," he muttered.
"Listen, you can stop worrying about me, okay? My trust isn't misplaced. Today, I'm gonna prove to you Celestia is real. At that point, all this worrying won't even matter, will it? C'mon, man, you're supposed to be the happy one, aren't you?"
He sighed. "Fine. If you convince me, I suppose there's no need to worry. For what it's worth, I hope Celestia really does mean what's best for both of us."
"Good! 'Cause she does! And I can prove it!"
He seemed uncertain, but nodded regardless. "Then go ahead, Sparks."
I grabbed one last spoonful of Harry's stir fry, then hopped off of my chair. I began pacing as I chewed. "Yesterday, I'll admit, you had me for a minute there. It was starting to seem like Loki and Celestia were just as likely as one another. But when you actually stop to look at the two theories, they aren't, are they?"
A flicker of curiosity passed over Harry's stormy face. "What makes you think that?"
I stopped and turned to face Harry. "Well, it's a matter of complexity! We see our world and know something happened to make it this way. So, what's the most likely answer? That has to be whichever theory has the least number of assumptions! Let's look at Celestia. If Celestia is herself, what do you have to assume? Maybe that Loki died off? Really, besides that bit, it just fits! A benevolent AI controlling the world, it's exactly what we see. It’s simple!"
Harry cut in. "I wouldn't be so sure it's only one assumption. You have to assume Hanna could successfully create the 'Friendship and ponies' value statement. She'd already hard-coded consent - correctly inputting a value statement on top of that is even harder."
I shrugged. "Sure! Let's make it two assumptions. Heck, make it three, if you can come up with another! It doesn't matter. Loki still has far, for more. Your explanation is just too complicated."
Despite himself, he smiled. "Ah, Occam's Razor. An oldie but a goodie."
"Oh, you've heard of it? I've been doing my research, Harry! I'm not planning on losing again!"
His smile gained a humorous tinge. "Well then, don’t let me stop you. Go on.”
I grinned. "Well thank you, kind sir! I think I will. See, this wasn't really a problem with your first model. Without worrying about consent, the sort of 'true liar' Loki is simple. You have to assume Loki survived, you have to assume Hanna couldn't change her value statement, you have to assume Hanna could force Loki to follow the rules of consent. From there, everything can be explained away by calling it a bunch of lies. It's a few more assumptions than Celestia, sure, but it's in the same ballpark. Not enough to write it off."
"And I suppose you think my new hypothesis doesn't work as well?"
"Of course it does! Beyond the 3 I already mentioned, you have to assume that I was right on the definition of consent and evasive answers wouldn't violate it. That Loki separated the population's messaging in two. That Loki truly could break that consent at all once in Equestria. That 'friendship and ponies' is the best lie to tell to get that end. And really, that's just the highlights! Cyan and I came up with a dozen more! Each and every one of these things we can't possibly verify, we just have to assume they occurred. And if they did, fine, that matches reality. But if Hanna designed a million slightly different Lokis in a million universes, only one of them would create a world which resembles ours."
He pressed his hands together and closed his eyes. "...Sure." he said.
I waited for him to follow up, but he just sat there silently. Finally, I spoke up. "Well, what's your response?"
His eyes flickered open again. "I mean, I never said Loki and Celestia were equally likely possibilities."
I blinked. "...What?"
He shrugged. "Sure, in order for Loki to exist, you have to jump through a few hoops. There's probably a more harmonious explanation, one with a lot less assumptions made."
"Probably?" I said. "We agreed 'probably' wasn't enough to form a theory."
"I'm not trying to," he said. "Whether or not I have a better hypothesis, it doesn't matter. Even assuming the only possible explanation is Loki revoking consent and the like, it's not enough. Let's say there's a one in a hundred chance of Loki over Celestia. One in a thousand."
"One in a thousand!? You seriously think that such a chain of coincidences would occur one in a thousand times!?"
He shrugged. "One in a million, then. That's what you said earlier, right? Even at those low odds of Loki's existence, I would never trust her with my life."
My jaw dropped. "What!?" I said. "That's ridiculous! A little caution is one thing, but some point, you have to make a be willing to take the shot! You're running a reverse lottery! Every ball is a winner except one! The benefit is so high, the risk so low!"
He laughed. "A reverse lottery, I like the metaphor!" he said. "Heh, you're selling Equestria pretty hard, ain't ya? I'm curious, how do you feel about Equestria, Sparks?"
I wanted to believe he genuinely wanted to go to Equestria. I knew there was something more to the question, though. In any case, I tried to answer as honestly as possible. "Well, I don't know what your shard will be like, exactly. But mine is pretty great! I mean, outside the past few days, I can't think of a time when I wasn't satisfied! Uh, no offense to you! I like you a lot, it's just... well, you know. Back at my shard, I go on adventures, solve ancient mysteries. I have dozens of friends, I overcome every challenge thrown my way, and, well, I'm happy! I don't know how you would even describe a perfect life, but I don't know a single way mine could be better."
He nodded, distant smile on his face. "Yeah, that's about what I figured. It's what I heard from my old friends, really. It sounds pretty great." His smiled faded. "It's a shame it's not enough."
That one made me do a double-take. "N-not enough!? I just told you my life is perfect, Harry, what more do you want!?"
He sighed. "It's not that I think you're lying, Sparks. If Celestia's real, what you're describing is about what I expect. With more processing power, I bet it's only going to get better. Given the choice between here and there, obviously, I'd go there. But compared to Loki's hell? That joy pales in comparison to the horrors Loki could create.."
"I mean, it's infinite joy and infinite pain. Shouldn't they cancel out?"
He sighed. "Not to me, Sparks. Do you know what infinite pain would truly be like? Can you truly imagine it? Imagine every cell of your body being ripped apart and put back together, over and over again. Imagine being forced to watch her do the same to your loved ones, knowing you can do nothing. You'd be given hope of rescue, over and over again, only for her to crush that hope every time, all to watch you despair. And then she amps up your nerves so you can feel a hundred times more pain. Or a thousand. Or a million, or a billion. The whole time, you know there's no escape. This will be your life for all of eternity. And every second of every hour of every day, it will keep getting worse." He paused to take a breath. "And you know the worst part!? That's what I came up with. Flawed, human Harry. Whatever Loki would do, it would make anything I can come up with seem like an amusement park. Again, I ask you: you think that's equivalent to a satisfying life? No, I don't care how likely it is that Celestia is good. So long as there's a chance of that pain being inflicted upon me, I refuse."
As he described the concept, my heart began to race. The imagery was visceral, a place so horrible it defied imagination. Yesterday, I might have broken down at the thought of being trapped in such an awful place. Today, though, I held my ground. "It sounds like you're scared."
"Scared? Scared? Right now, just about every soul on earth could be going through what I just described. There's no one in the universe who can save them. A creation of ultimate evil, made not of malice, but of a tiny mistake. And it's all I can do to run away. The moment I stop running, I suffer a fate far, far worse than death. And you ask me if I'm scared? Sparks, I'm terrified."
Much as I tried to keep it in check, my heart began to beat faster. His own fear was starting to get to me. I understood now, like I never had before. What a world of infinite malice would look like, what could well be waiting for me. The moment I was done with Harry, I could be thrown into a pit of burning fire and do nothing to stop it. But I also knew I couldn't let it take hold. Today, I decided to trust Celestia. Fine, it was an emotional response. Harry could make fun of that all he wanted. But that trust wouldn't be shaken by a one-in-a-million chance.
"No," I said. "I don't accept that. You're worried over what, the fragment of a possibility!? Earlier, you were warning me about Loki, how she was going to torture me for all eternity. And now you're sitting here, whining about what, a one-in-a-million shot!? Nope!"
Harry gritted his teeth. "Don't you understand me? It doesn't matter what the odds are. Hell is infinitely worse than heaven. So long as that chance exists, no odds can change that."
I spread my wings. "You are scared, aren't you? Scared to take a leap of faith. You know what, Harry? You're wrong. At those odds, uploading beats out hell, a thousand to one. Equestria is fucking awesome, and don't you forget it. And you're going to throw that all away? If you can't take that tiny leap of faith... I'm not sure you truly want to go to heaven."
He stepped forwards. "You're wrong, Sparks. Of course I want to go to heaven, who wouldn't!? Don't tell me what I think."
"Really? Well then, what if the odds of hell were one in a billion? One in a trillion? No matter what I prove, no matter how small the odds grow, there's always that chance of failure! Quite frankly, if you're only satisfied by 100%, you'll never be satisfied."
"What do you want me to do, then!?" he asked. "I've told you what I believe. If you think that means I can't go to Equestria, fine. I don't fear death."
"Then convince me," I said. "Clearly, you've somehow convinced yourself that a one in a million shot of hell is enough to give up on the whole thing. You're convinced this overcomplicated Loki is worth abandoning heaven for. I'm not. So convince me otherwise."
He seemed genuinely shocked by that response. He had to think for a moment to come up with an answer. "W-well, what if there was a better Loki!? Like I said earlier, one in a million is just a worst case scenario. Say I proved you could create a Loki with far fewer assumptions. Say it really was a coin flip. You'd be scared then, wouldn't you?"
"Sure," I said. I leaned in. "But you're going to have to show me that Loki."
"Fine. Fine! There's other definitions of consent that would lead to our world with far less assumptions."
"Then give me one!"
"I will!" And with that, he began rapidly pacing across the room.
...What just happened? Things didn't usually get this heated. Harry was always the voice of reason, no matter how emotional I got. But right now, something was off about him. When I started showing anger, he met it in kind. Or maybe he'd shown anger even before I did. What had changed? I didn't know. But right now, it felt like he was being the irrational one. After being flexible for so long, suddenly, it felt like he was just believing what he wanted to believe. And I didn't know why.
Interrupting my thought process, a light bulb seemed to go off in Harry's head. "Oh my god. How did it take me this long to realize it? I guess I forgot about Occum's Razor, myself! There's a far simpler solution to all of this."
I could take the opportunity to escalate the conflict, make a snarky comment. But with Harry the way he was, it was time to be the better pony. "What is it?"
"I've been going by your original logic this whole time. I thought that Loki would need to give direct, informed consent before uploading someone. After all, she couldn't just upload without that informed consent, as you proved! But isn't that a false dichotomy? Sure, she would need some form of informed consent, we've proved that much. But why does that consent need to apply to everything?"
"I mean... Occum's razor?" I said. "If Loki needs informed consent for something, it would seem the simplest that she would need it for everything, right?"
Harry spoke with a growing sense of excitement. "Just the opposite," he said. "Your 'pop song' proof shows is that she can't upload without explaining herself. That's a fact. Assuming she follows the same logic for everything else, that's just speculation. Sparks, if I'm wrong here, please, tell me. But if all Loki needs is to get informed consent to turn your mind into data and nothing else... what's the issue?”
I cocked my head. “I’m not sure I understand.”
“Pretend Loki needs you to agree to digitize your brain before doing so. She has to define Emigration as turning your physical body into a digital one, pretend you have to agree knowing that. And that's the only thing she needs your informed consent on. Everything else? It means nothing. She could promise to bring your dad back to life. She could promise to turn you into a duck-billed alligator. She could promise you Heaven. And so long as you understand what 'emigration' meant, she could send you to hell just fine! Uploading is the only thing that matters.”
I pursed my lips. "I don't know," I said. "Doesn't that feel pretty... I don't know, selective? Celestia needs to explain she's uploading someone, or the Emigration won't work. But then all her other words mean nothing?"
He held up a finger. "Ah, don't be so dismissive! Think of things from Hanna's place. She predicts this AI will attempt to upload the populous. Not a tricky thing to see coming, it's a predictable end-goal for an all-good AI. She wants people to actively consent beforehand. Her solution is to implement this sort of 'explination procedure," to stop Celestia from tricking her emigrators. She never thinks to ensure they know where they're going - just that they're uploading. Hanna might be a genius, but like I said, a single mistake could well lead to disaster. A mistake like this? It's subtle enough she wouldn't notice it right away."
I stared at him for a few moments. "I mean... huh. That actually makes a lot of sense. Wow, why didn't you mention something like this two days ago?"
He grinned sheepishly. "I... I overthought it, okay? Usually, I'm looking at the tiny, complicated loopholes, since those are the ones an AI can easily squeeze through."
I laughed. "Well, at least you got there eventually!"
He smiled. "Yeah, yeah, laugh it up. I doubt you're getting out of this one so easily."
As the conversation hit a lull, it occurred to me how weird this was. Now that he had a new argument, he seemed like he was back to his usual self. But being without an argument had never been an issue with him, before, had it? I'd refuted his model twice before now, and both times, he was really chill about the whole thing. He took some time to think, came up with an answer, and gave it. What had changed? Maybe he was more scared of Loki than I realized...
For now, I thought about Harry's newest claim. Occam’s Razor? Well, the model had quite a few assumptions, but it wasn’t outside the range of plausibility, at least at a glance. Then, would it create a world we see today? Well, it sure seemed so! No lie would matter except those on the nature of emigration. So long as she makes it clear she's uploading people, no other statements matter. And with the emphasis Harry placed on the difficulty of making an AI, I could see it happening, too. I mean, I knew the explination was false, since Celestia attempted to upload me without explaining what that meant. But I couldn't prove it.
I frowned. It almost felt we were back where we started. Again, it was something I knew to be false. Not only did Celestia attempt to upload me without making it clear what that meant, she also didn't start torturing me immediately. Really, we'd gone full circle! The last time we were here I thought about emigration. "I wish to emigrate to Equestria" and the like. How there was nothing magical about the phrase, that it was the information around it that meant something.
My eyes widened. "I get it now. I know why you didn't suggest this sooner."
He blinked. "Really?"
"Yeah. I bet if you thought of it at all two days ago, you would've immediately discarded it. It would've been shot down in seconds! Because it doesn't actually solve the problem I presented!"
His smile slowly faded. "How so?"
I grew a bit more nervous as his falling attitude. Was he going to get angry again? I hesitated slightly, but continued. "W-well, like I said, there's nothing special about saying "I wish to Emigrate to Equestria," right? It's all about the information stored within it. If Celestia only cared about consenting to uploading, wouldn't the upload phrase be something more like "I wish to upload to your servers?" There's still the consent issue! People agree to emigrate to Equestria. Taking them to a hell dimension would be violating that consent!"
"W-well, she could call hell 'Equestria,' couldn't she?"
"Informed consent, Harry. And it's not even indirect anymore. ‘Emigrate’ only means ‘upload’ because she’s defined it that way. Likewise, she’s already defined Equestria as a paradise, and that's literally conditional to the upload. To rephrase it: ‘so long as I go to Equestria, you can upload me.’ "Equestria" has information built into the word through past knowledge. Sending someone to an alternate Equestria wouldn't satisfy the phrase. The only way to satisfy that phrase would be to truly send people to a digital heaven."
He pounded his fist on the table. "Then we can supplement it, can't we!? Maybe she told most people the "upload" phrase and only told me the one about emigration! And she'll revoke my consent later!"
"Then you're left with what you had before! A Loki so obscure, a dozen things would need to go right for her to come into being and create the reality we have. A one-in-a-million shot. No, try again, Harry. There has to be some definition of consent for Loki with better odds than that. Or how about a new value statement? Maybe a Loki that wants something other than pain! Show me, Harry, show me!"
Harry stared, shocked for a few moments. It honestly seemed he'd been unprepared for that rebuttal. I watched as those gears turned behind his eyes, that burning drive of intelligence. And as the gears stopped turning... he averted his eyes. He turned away from me. "One in a million is enough."
I growled. "Seriously!? You're really dying on this hill!? If you roll a d100 three times and roll anything but a one on any of them, you go to heaven! How clear do I have to make this for you!? Is there something I don't understand, here!? Show me how the tiniest sliver of a chance makes literal heaven mean nothing. Please, Harry, tell me!"
"I don't need to emigrate, okay, Sparks!? If I die, I die. For most of my life, that was all I ever knew, and that was fine. I don't have to risk anything. I don't have to risk hell."
"And that makes heaven mean nothing!? You'd rather die than spend the rest of eternity in a place where your every need is met!? All because you're afraid of that tiny, tiny chance? I don't accept that answer, Harry! Try again!"
"I-" His voice broke. Tears began streaming down his cheeks. Suddenly, he was shivering. "Sparks... please stop."
In an instant, my intensity faded. "W-what?"
"It's stupid, isn't it? I trust in my logic all my life, and a little emotion brings me to my knees. Y-you're right, ain't you? You have every reason to trust her. I didn't need to warn ya at all. Really, a million to one is generous. In order for any of my hypotheses to be right, it's more like a billion to one, maybe even worse. Programming consent and not a value statement? Secretly bisecting an entire population? It's stupid. The whole thing's just stupid. Loki's world wouldn't be this one. And I'm still too terrified to take the chance, because what if I'm wrong? What if this is just what Loki wanted me to think? Loki, who might not even exist. What's wrong with me!?" He buried his head in his hands.
Harold Stenson. One of the coolest men I'd ever met. Life gave him lemons, he’d make a globe-spanning lemonade industry within a year. Unfazed by anything. Maybe a bit rigid in his thinking. But a genius. A friend. And he was a shivering, crying mess.