//------------------------------// // 5: Haymaker // Story: Friendship is Optimal: Lies // by MLfan //------------------------------// When I opened my eyes, I found myself not in a field, but directly in Harry's house. Harry was standing by the burner, in the middle of cooking a bit more stew. "Ah, Celestia! You're a bit early!" My last conversation with Celestia still danced in my head. I was right: she had lied about how urgent the mission was. Harry wasn't in any sort of danger. So why? If things weren't time-sensitive, why pull me out of the abandoned city the way she did? Was there some other time limit I wasn't aware of? Why else would she pull me out at such an awkward time? The harder I tried to figure her out, the more unknowable Celestia became. She could try to convince me otherwise all she liked; I could only ever see her as a god. A god who lied as easily as she breathed. With that in mind, I supposed it was easy to understand why Harry thought she was Loki. What was another lie to add to the pile? Speaking of, there was Harry, right in front of me. And he'd just called me Celestia. "Alright," I said. "Let's set some ground rules. You can call me "Sparks" and "Promise." I don't give a buck what you think my name is, those are the only names I'll respond to. Do I make myself clear?" A slight smile. "And if I don't?" "Then I leave. You spend another day outside of heaven, and you spend another day being lonely. Because guess what, asshole? I'm not Celestia! If you keep being a jerk to me, at some point, you getting into heaven isn't my problem anymore. Do I make myself clear?" The smile grew on his face. "Loud and clear, Sparks. Man, it's good to have ya back! Honestly, I preferred you to your Celestia persona from the moment I saw ya." Well, he still thought I was an alternate Celestia. At least he was using my name? At the very least, it was a start. He smelled the stew and smiled. He began scooping into a bowl. "Sorry I didn't have it set out for you when ya got here, by the way. I hoped to surprise you with it. But like I said, you were early!" I blinked. Did he just say what I thought he did? "I thought the soup was just for you," I said. "Did you really prepare an entire meal for an AI?" "Eh," he said. "Why not? It's only polite, isn't it?" A small smile grew on my face. "No, no, no. That's not just "polite." That sounds to me like a pretty big waste of resources to me. Yesterday, weren't you oh so certain I was an extension of Celestia? What, does some secret part of you know I'm real?" He chuckled. "Eh, maybe. Like I told you yesterday, it's a convincing illusion. If that part of me exists, though, it's not a particularly smart one. Besides, I have plenty of spare resources." And yet he didn't stop making that stew. He had his doubts. I was so damn human he couldn't help but realize it, at least in some small part. "Yeah, sure." I said. "Keep telling yourself that." He grinned. "Oh, I love the cockiness. This is gonna be a fun one, isn't it?" I leaned in. "I sure hope so." With a flair, Harry slid the bowl down the table towards me. I caught it just before it went over the edge. "Then let's get started," he said. He took a seat of his own. "Let me guess: you're gonna pretend you don't know what Celestia told me yesterday?" "Not pretending." "Sure, I'll fill you in. Because she did something strange. She gave me a hint only I would understand, to point me towards the argument I'm going to give today. Trust me, it's pretty damn airtight. Why do you think she'd do something like that?" It was some sort of challenge. He probably thought he was fishing for information from Celestia herself. Unfortunately for him, I didn't actually know anything. "And let me guess, you think I have some sort of hidden insight as to how she thinks? Nice try. Until I hear this shiny new reasoning of yours, I don't have a lot to go off of." "Trust me, once you hear what what Celestia was hinting at, you'll have bigger things to worry about. In the meantime, how about you give me a guess. Isn't pattern recognition a particularly human trait?" Oh, it was on. "Well, maybe your argument isn't as clever as you think it is, ever think about that? Celestia might just be leading you into a trap, and it's just as flawed as your last one. Or maybe she wanted to earn a few brownie points from you by seeming nice. What about both? She's a god, I doubt there was only one reason for it." He eyed me for a few moments. "Fascinating," he said. "I'd thought of a lot of the exact same things. But then you call her a god. Now why would you call her that? You've never asked me to call you one when you were Celestia, herself. Why change it now?" I rolled my eyes. "And I'm supposed to be the one who's Celestia in a trenchcoat? Sheesh, she said the exact same thing to me like 5 minutes ago. Just 'cause she asks not to be called a god doesn't mean she isn't. That's not how definitions work. Complain all you want - definitely, you're still a human. And definitionally, Celestia's still a god. Unlike her, I'm just not afraid to admit it. And for the record, I'm giving different information than Celestia 'cause, y'know, I'm a sapient being who can make my own decisions." "Subtle," he said. "I suppose it doesn't really matter. I was just curious." Hm. I could’ve just moved on, but... "Who is she to you?" I blurted. Harry clearly didn't see Celestia as a god like I did. After the conversation I had with Celestia, I could help but be a bit curious how others perceived her. And out of anyone I could ask, Harry’s would definitely be the most interesting. "Honestly? I think she's an object." I blinked. Yeah, interesting was one way of putting that. If you’d given me a thousand guesses, I'm not sure I'd ever come up with that one. "I think I'm gonna need a bit more context on that, Harry." He shrugged. "Do you? I think I was clear enough. Quite frankly, she might as well be a brick wall." I let out a confused laugh. "Yeah, but like, she isn't though? She can talk, she can think, she has a face. I don't think it's particularly controversial to say she's clearly alive." In response, he rolled his eyes. "Fine then. How about a brick wall with glitter and googly eyes? Sparks, she's a program designed to maximize values or pain or whatnot. For any given input, she gives an output that creates the highest growth to that function. Any emotions she shows, any things she says, they're just the things the function predicts the best outcome out of. If she smiles with a human-like face, it's to put someone at ease, not because she's actually happy. At least from my point of view, that ain't alive." That... made sense, actually. I mean, I didn't really agree with the conclusion, but I understood the logic behind it. Leave it to Harry to spin a compelling argument out of such a ridiculous conclusion. "Huh... you know, I've never really thought of her like that. Personally, I think she really is thinking under the hood, if very differently than you or I. Maybe we could debate on that once you Emigrate." He leaned back. "Yeah, I think I'd like that." He shook his head. "Man, it's nice to chat with you, Sparks. If you were real, I have the feeling we'd get along pretty well." You know, I’d almost forgotten he thought I was fake. In response, I flashed a sugar-sweet smile. "Like I said, when you Emigrate, I'll be there, and you'll see just how real I am. You know what, the sooner the better. How about we get your flimsy logic out of the way? Somehow, I don’t see you proving Equestrian isn’t heaven.” He smirked. “Did I strike a nerve, Sparks?” “Ready to lose again, Harry?” "Oh, I'm looking forwards to it." He leaned in. "So Celestia gave me a hint, for whatever reason. She reminded me that "happily ever after" isn't always the ending it seems to be. On that note, yes, I've thought over your logic from yesterday. Quite frankly? It's flawless. Without informed consent, I would've gotten into Equestria, already. I wouldn't expect any less from a perfect AI. So I concede! Equestria is a place akin to heaven. That is... at first." "Yeah? You say that like it might change." He grinned. "Pretend, for a moment, you're Loki. So difficult to imagine, I'm sure! And let's say you've invited someone to Equestria, promised them their values would be fulfilled for as long as possible. They've said yes. What happens next?" I scoffed. "Come on, really? You think Loki would ever let that happen? If I was Loki, I wouldn't invite them to heaven in the first place! I'd just find a way coerce someone into hell without the heaven middleman." "And if you can't find a way?" "Then I guess I'm not Loki, then, am I? Come on, Harry. She's a god. You think programming restrictions could seriously stop her from sending people to hell? If literally creating heaven really was the only option for uploading, I doubt she'd upload people at all. How hard is it to claim she's sending people to a world where she 'has our lives under her control' or whatever? " He let out a sigh. "Okay, fine, then. You want to stick your feet in the dirt, I can be flexible. Imagine that Celestia uploaded someone, promising them satisfied values for a maximally prolonged period. And then, Loki suddenly switched positions with her. But for whatever reason, Loki is bound by the promises Celestia made. What would Loki do then?" That question was a bit more interesting. I had to stop and think about it. No matter the angle I checked, though, I wasn't sure if I really saw a way out? Like, she had promised that Equestria was heaven, that values would be satisfied for as long as possible. Under that assumption, Emigration occurred. In a way, those promises were just like my own conditional Emigration request. When I Emigrated, there was the promise that so long as I was allowed to die, I would Emigrate. Celestia said it was like a new line of code had been written for her. Loki couldn't break that promise, because if she did, Emigration could have never occurred. So she really wouldn't have a choice but to create heaven by her very programming. "...Huh. Come to think of it, I'm not sure there is a way to torture Equestrians. Whatever actions she took, they would need to be to create maximal satisfaction for residents of Equestria. If Loki somehow promised pain for eternity and Celestia switched, I think she would have no choice but to do the same, right?" He chuckled "Personally? I think it's a good thing she gave me that hint and not you. You lack the imagination of an AI." "Oh, really? Then I suppose you have a foolproof way of circumventing the rules, then?" "It's simple, really. Her rules for consent don't change in Equestria. And any consent given can still be taken back." My body tensed. A fork I'd been holding clattered to the table. In an instant, I was thrown back so many years, to a time when I was only barely used to Equestria. I hadn't even met Cyan Skies yet. "I no longer hold you to my promises." ...Shit. Shit, shit, shit! The promise Celestia made to me, it lasted only a month. And in the end, she broke it. Or... she convinced me to break it. I did so willingly. And it was practically effortless! If Loki suddenly took the reigns on Equestria... What if she already had, and I hadn't noticed yet? What if I'd given away my freedom!? What if I didn't just give away my right to die, I gave away everything? Harry's voice snapped me back to reality. "You okay, Sparks?" I was breathing heavily, and my whole body was shaking. I frowned and picked up my fork. Stupid brain, I had no reason to react like that. If Loki was going to do anything to me, she would've done so already. How was she supposed to suddenly switch with Celestia, anyways? "Just peachy,” I said. “Look, it's nothing." "Sparks..." I scowled. "Drop it. I'm fake, remember? If I’m freaking out, that’s just me keeping you on your bucking toes. Wow, look at how funny it is that you have concern for a robot! Celestia invented a fake backstory so you'd feel sowwy for me. Wow, don't you feel so stupid." I flashed a mocking smile. The statement hit him like a punch in the face. The longer I talked, the more and more shocked he looked. "I-I didn't... I mean... but..." he trailed off. I took his moment of confusion as time to cool myself down. Yeah, that was too harsh, wasn't it? I was reminded of a stressful time, but that didn't give me the right to start lashing out at everyone around me. I had an overreaction. Celestia wasn't Loki, I already knew that. I shouldn't have let personal matters get in my way. "I'm sorry," I said. "Just... hit a nerve. That's no excuse for lashing out, though. I should know by now you don't actually care." He took a shaky breath. "I... I hope I can, someday." I sighed. It was a fine sentiment, but all it meant was that right now, in his eyes, I was nothing. I turned my head away. "Whatever," I said. "Listen, Sparks. If you're real... I'm sorry, okay? I don't want to hurt you. I just... I don't want to be manipulated. Because if you are fake, then those are just fake tears, and suddenly I'm the idiot. I guess it's like you were saying yesterday, you know? If I slip up for even a moment, Celestia wears me down further. I'd never really put it into words, but... you're right." I nodded slowly. It was hard to stay mad at him. After all, he was just like me in the end. He was distrustful and lonely, even if he expressed it differently. "Yeah," I said. "I understand." I let myself smile. "Just so you know... it's not gonna last forever. It didn't for me. So stay doubtful. Make every effort to make sure Celestia is who she says she is. But when you see the truth... she'll be there for you. You won't have to worry about being lonely anymore." He smiled at me. "I'd like that, Sparks." I took a moment to grab a bite of my stew. "I think I'm ready to continue if you are. Again, I'm sorry for freaking out like that. I'm supposed to be here to help you, you didn't ask for all the baggage." "Nah, it's no hassle. Like you said yesterday, you didn't ask to be here. Assuming you ain't lying, of course." "Then go on. I think I interrupted you." He took a moment to remember where he was. "Well, I think just about got the gist. I think Loki might have brought people into Equestria, then manipulated them into taking back their consent." Okay, then. It was about what I thought. Really, his argument hadn't really changed all that much from yesterday. Loki was masquerading as Celestia, claiming she would take people to heaven. Difference was, instead of dumping them in hell, she would take a pit stop in heaven until she tricked them into dumping their promises. It was a scary idea, to be sure. But after I thought it through, it wasn't all that compelling. The argument might have been similar to yesterday's, but the circumstances behind it were all different. "Sorry to burst your bubble," I said, "But I'm afraid your new theory is just as flawed as your last." "Really? You didn't think so the first time you heard it." I tensed slightly, but took a breath and continued. "You surprised me, okay? Really, though, it only works inside of that hypothetical." "Now what do you mean by that?" "Don't you remember the first thing I said when you told me the premise of your argument? Unless you reach a situation where Loki has made a bunch of promises of infinite happiness, this whole thing is a non-starter. Because if you ask me, Loki would never promise a realm where she satisfied peoples' values. Fine, if Loki suddenly jumped into Celestia's body, she could make a workaround. But why would she need to?" Harry’s face burst into a grin. "And as the referee blows the whistle to resume the match, Sparks comes out with a right hook!" He cracked his fingers. "I'll admit, that's one of the biggest sticking points, there. Let me ask you a question, though: how, exactly, would you suggest Loki do better? Remember - she's bound by consent. Any mental modifications, up to and including Emigration, need the informed consent of the participant. It's a bit harder to come up with an answer than you'd think." "I mean, couldn't she just say she could upload people and leave it at that? Show the wonders of Equestria Online and say 'I want to take you into my virtual world.' It implies she’s taking them to to Equestria Online without actually saying it.” "Ah, but what about followup questions? 'Can you really make me happy?' 'Will my friends be there?' And for the sceptical who understand AI, how about… 'What is your true value statement?'" "Then answer evasively. 'Yes, I can make you happy.' 'Why yes, your friends will be there.' And how about... 'I was programmed to maximize human values through friendship and ponies.' No lies needed, since in your scenario, she was programmed that way! Oh, and the ominous undertones I'm voicing would be omitted, obviously." "Hm... that might work for some. But can she keep up the lie on live TV? A lot of people will notice how evasive those answers are. And what if someone gives a question she can't answer. What about... "What can I expect Equestria to be like?" "You'll spend every day surrounded by friends, with new experiences beyond every horizon." He put his hands under his chin, mimicking an overexcited schoolgirl. "Oh really? Tell me more!" I held out a hoof and puffed out my chest. "I'll always be watching over you, making sure everything goes perfectly. You'll never be alone again." "Come on, you have to give me more than that! What will my days be like, personally!" "Oh, I can't say until you've emigrated and scanned your brain what it will be like, exactly. I could give you another Emigrator's experience, however." "So you're saying Celestia would choose not to torture some ponies just to trick the rest? Oh, that's clever. How about... 'Show me my family!'" I twirled around. "Why, certainly! It wouldn't be hard to keep a family member or two in paradise to be on the safe side. 'This is the average day of your mother. She's been waiting for you.'" "And that's what my days are going to be like? Forever?" "Celestia doesn't answer. She just gives a knowing smile and lets you fill in the rest." The two of us stared for a few moments. Then, we both burst out into laughter. "Alright," Harry said, "It's an effective decoy, I'll give you that. Though, I do have to wonder if it would actually work in reality. Loki might not have actually said she'd be putting people in paradise, but she's absolutely implied it. Wouldn't that mean the consent isn't truly informed? At that point, taking someone to hell isn't that far off from taking them to hell after having them sing a pop song." "You know, I thought about that," I said. "I don't think it's so clear-cut. It's a long story, but when I first agreed to Emigrate, I didn't even know I was agreeing to do so, I just agreed to travel to Equestria to be with my friends. It wasn't until after I refused, much later that she explained what 'Emigration' was. She deceived me, but still could have emigrated me. To me, at least, that shows she doesn't necessarily need to tell the whole truth - just enough for it to count." "Assuming you're telling the truth," he said. "Fair enough," I said. "Still, even if my example isn’t perfect, I think it proves your theory has holes in it. Loki might not create my world, exactly, but I don't think her actions quite match up with what we see on Earth." He leaned in. "You’re making some assumptions, yourself, y’know. Sure, if you're right about what Loki defines as consent, she might not need to lie about heaven. But we can't know that for sure, now, can we? I mean, I can never tell why Celestia does what she does. For all we know, her definition is such that creating a world like this one is her best option." Well, he had me there. I knew Celestia about as well Harry did. I could hardly begin to understand half of the things she did. It was an honest answer, all things considered. He was right, there very well could be some context where some version of Loki could create our world. Still, it felt so odd. An AI bred for pure evil sending people straight to heaven upon uploading. If he was going to say that, I needed more than a 'what if.' So, I spoke. "For all we know, hm? I'm not so sure that's good enough." "Oh, really?" I tilted my head thoughtfully. "Well, you've presented two theories. One of them clearly fits with the world around us. If Celestia is who she says she is, we don't have any issues. As for your counter-narrative, that Celestia is Loki in disguise? It isn't complete. A Loki that doesn't care about consent would have Emigrated you already, and a Loki that does wouldn't have promised you heaven in the first place. What's your solution, then? You're trying to make it work by filling it with 'maybe's and 'I don't know's? Maybe Loki has a definition of consent that requires her to make this world? Nah. Unless you have a road to the world we actually live in, it's not a valid theory." Harold mimed recoiling and wiping his mouth. "And Sparks gets in a laft jab dead onto his face. Will Harry remain on the backfoot?" He rose out of his chair to face me. "You know what, you're right. I can't hide behind 'what if's forever. Give me a moment to think, Sparks." He began pacing across the floor of the house. Hey, I was doing pretty well for myself! He'd seemed genuinely surprised by my arguments, so much so that he had to stop and think! Compared to yesterday, I'd been able to lead a lot of the discussion up to now. Plus, the comments on me being Celestia had almost completely fallen off, so make that another win for me! Still, Harry was smart, incredibly so. I didn't doubt he'd be able to turn a bad situation around. Suddenly, as if on cue, his eyes lit up and he faced me again. "Wait a minute... oh, that's wonderful! You want a new hypothesis? Let's use yours." My ears perked up. "I'm listening." "It's simple: who's to say we aren't living in the world you're describing? She chooses her words carefully to trick people into believing emigration leads to a sort of heaven. Equestria Online is used as a sort of lure. It appears as a sort of paradise, so people assume emigration will be much the same. Her answers are perfectly evasive, to the point where it's almost impossible to figure out Equestria isn't heaven. I even have an example! Remember, the first time she spoke you, Celestia didn't actually tell you she would satisfy your values. According to you, she just said she would take you to Equestria. If you'd said yes, that would be it!" I scrunched up my face. "I mean, that's great and all, but what about you? From what you've told me, I'm pretty sure Celestia has unambiguously told you her value statement and whatnot. After my initial encounter, she definitely used precise language around me. Trust me, I was really paranoid at the time. I would've caught some grand conspiracy." He held up a finger. "Ah, but that's when we loop back around to my original hypothesis! If she's forced to say Equestria is a place akin to heaven, what's to stop her? Someone gets too suspicious, she tells them exactly what they want to hear: she wants to satisfy their values for as long as possible. After all, she can always run it back later! Sure, she might give up a bit of pain in the short-term, but there's still a near-eternity of pain to go once she's broken down those barriers. Some go to hell right away and some wait, but we all make it there eventually." I had to admit, it was a compelling narrative. To use his terminology, it was the first swing he'd made that really landed. The model was complete, and even used my own experiences against me. I thought carefully. Was there any part of that narrative that clashed with reality? Personally, Celestia had told me many times that she intended to satisfy my values in Equestria, and Harry would probably be the same. But was it possible those messages were targeted only to me? ...No. No it wasn't! A grin spread across my face. "Sorry to burst your bubble, but your new theory has a few holes in it. I'm sure you've forgotten now that the world's fallen apart, but Celestia's messages weren't always as personal as they are now. She released ad campaigns, televised speeches countrywide. You couldn't go anywhere online without there being memes of the repeated phrase 'Satisfying values through friendship and ponies.' Private deception is fine and all, but she wasn't just telling the few who would see past the obscure language. She was telling everyone." "Oh, it sure looked like she was, didn't it?" He grinned. "But couldn't that just be an illusion? Remember, Sparks, Loki's a highly advanced AI. She could create untracably different TV signals to give different information to different people on the same channel. In one, she gives direct language, promising friendship and ponies, and on the other, she only implies it. She can direct advertising on a person-by-person basis. And online chatrooms are easiest of all! Celestia's a digital being, she can warp anything you see on there as she pleases!" This time, he really did punch the air in my direction, accentuating his statement. Damnit! He'd turned my entire model on its head. All of a sudden, I was on the backfoot. It really did fit everything. A world where she only lied to a select few would definitely fit within Celestia's abilities. It was absurdly flexible, really. It leaned into the massive, far-reaching lies that I knew Celestia had the power to make. I didn't doubt that any small holes in the theory could be filled with minimal effort. Worst of all: he'd created a scenario where revoking consent mattered. This whole time, I'd been arguing Loki would never let a world reach that point. I'd been avoiding the whole thing. Because, well... what was I supposed to do to argue with it!? I had taken back my own consent post-emigration. I couldn't argue it was impossible, because I knew it wasn't. Celestia, if I'd known about all this, I would've been a bit more careful with my wording. Sure, it turned out to be fine, Celestia didn't suddenly start torturing me or anything. But I should have been more careful. I could have agreed to break from my suicide option while ensuring she kept her promises about paradise. The moment the thought crossed my mind, my eyes flickered towards Harry. "Wait a minute. Why do you even care whether Celestia is Loki?" "Does 'because I don't want to be tortured' count as a reason?" "No, no, no. I mean, if you're so worried about Loki, couldn't you just... not agree to her demands? Like, if Celestia asked you to revoke the promises she made you, couldn't you just say no? So long as you don't revoke your consent, she'll have no choice but to satisfy your values!" "While everybody else is being tortured? Doesn't that seem a bit... selfish?" I blinked. "Uh, I guess? Uh, I hadn't thought of that. You hadn't really talked about taking down Loki or anything until now. I don't know, if you somehow found out that AI is definitively Loki, are you going to start gunning for a killswitch or something?" He eyed me for a few seconds, then broke out into laughter. "Well touché! I guess you're right. Loki or Celestia, either way, she's too far along to be stopped at this point. If Loki is torturing billions of ponies, there's quite literally nothing I can do to change it! I s'pose it makes sense to live your best life, eh? Unfortunately, if I'm dealing with Loki, I don't have the luxury of taking advantage of her kindness." "Why not? It should be easy enough to tell her no if you know what she's asking for." "Oh, but now you're forgetting what emigration truly entails! Let's say I emigrated to Loki's domain. How many times do I have to turn her offers down, do you think? Let's say to lull me into a false sense of security, she doesn't even mention it for the first ten thousand years. Gains my trust all the while. Then she asks, 'Hey, Harry. We've been together this long. Do you trust me?'" "And what, you think she's gonna get you to somehow revoke all your promises you made on earth" "After ten thousand years of heaven? Yeah, maybe. Before Celestia, humans didn't live all that long. A hundred years if we're lucky, and by the end we barely remember who we are. After ten thousand years, I don't think I'll be the person I am today." "Then write a notebook to yourself or something! Every day, remind yourself never to give Celestia an inch." "Then she comes back in another ten thousand years. Then a million. Sparks, how long do you think I can hold out? Celestia is a patient mare. That time is nothing to her. And whenever she breaks me, that's still trillions upon trillions of years until entropy takes me. That million years of joy could be rounded down to zero." ...All he had to do was say 'yes' once. And his life would change forever. He could win a thousand times, but if Celestia wins even once, it will all be over. That was the exact thought process I had for why I was afraid to emigrate. It was why I tried to let myself die. The idea repeated in my head, again and again and again. Why couldn't I figure this out? Celestia wasn't evil. She couldn't be! So why did she keep acting like Harry said Loki was supposed to!? There had to be some thread to pull, something that would make this whole thing unravel. There had to be, because Celestia wasn't Loki. I wasn't in hell. I wasn't! Celestia must have laid out some plan for me, some hint to show everything would be okay, that Equestria was the paradise I knew it to be. So why couldn't I find it!? My body began shivering. "I have to go," I said. He flashed a look of confusion my way. "What? But we've barely started, Sparks!" "I HAVE TO GO!!" I snapped. I closed my eyes and took a breath. "I mean... I don't know why you're wrong yet. Please, g-give me time." "Sparks?" he said. I hadn't noticed him walking towards me, but suddenly, he was next to me, reaching a hand towards me. I flinched backwards, then took a few quick breaths. "I'll see you tomorrow, Harry." Before he could say anything more, I used a wing to slam my legband, and the world erupted into a rainbow light.