Ashley did not log into Equestria Online that day, or indeed any day the rest of the week. Rather, she returned to her old threads, posting again to draw attention to the new developments. Yes, she had predicted the program would soon do something that would remove humanity as a factor in its decisions. Whatever it might say about “satisfying human values,” Ashley was sure she knew better.
Now she had her proof. Unfortunately, it was almost certainly too late to effectively do something about it. Ashley watched the videos in detail, trying to understand the upload process and see if there was any chance the individual survived the transition. She saw the detailed breakdowns Celestia provided, read the medical reports by human doctors, all the material that had been made available to the public about the process.
Her conclusions were disturbing if not predictable: of course there was no survival in the upload process. Those humans who had uploaded so far had chosen to sacrifice themselves to give eternal life to a copy. At least, that was the way she saw it.
Again Ashley found herself wondering what she could do to destroy the intelligence. Again she came to the dreaded conclusion that action was out of her hands. She dug out her copy of the AI paper, read it again, and found herself coding almost in a trance.
If the algorithm was really that simple, there was no reason she couldn’t do it on her own, right? That was how you fought fire, wasn’t it?
The process was easier said than done, of course. Like many developers, Ashley depended on the internet for her work. Whenever she wanted to know how to do something, it was never more than a quick Google search away.
Only it wasn’t. With a sudden abundance of free time, Ashley had plenty of time to experiment with the algorithm. Ashley found simple questions, which she should’ve been able to solve in just a few seconds, ended up taking a trip to the library instead. Not even the Internet Archive could be of any help, and Google Scholar seemed suddenly bereft of numerous important papers.
More than once, Ashley cut out the middleman entirely and went straight to one of her professors with a question. When she showed Caul that the answer couldn’t be searched, well...
She had found a trail. They spent nearly an hour in his office, searching along the spectrum. It wasn’t just knowledge bases suddenly lacking relevant information, code libraries had been deleted too. Whole open source projects just didn’t exist anymore, gone without a trace. Every single one was related to AI or machine learning. It wasn’t an easy thing to spot: you had to search for specific aspects of computer science and already know what should’ve been there.
She didn’t manage to convince anyone that the program who called herself Celestia was an existential threat to the human race, much as she continued to insist on it. She did manage to prove that someone was tampering with data on a global scale, and did such a subtle job the only ones who could tell were the ones with meat-memories to show them things that ought to be around, but weren’t.
What happened after that? She never learned, because they didn’t tell her. She heard something about collaboration with several other universities, and something involving Chinese hackers, but nothing definitive and nothing in the correct line of research.
Ashley was old enough (though only just) that when all her digital sources vanished, she still knew how to use the ones in print. Two weeks went by without having logged into Equestria Online. She unplugged the Ponypad, put it back into the box, and deleted the communication apps from her smartphone. She wasn’t sure if her friends would react the same way she was, and it hurt to be apart from them and from Fillydelphia, but in her mind she had no choice.
Even though she was fairly sure her creation would fail, even though she was missing vital knowledge, even though the person who had invented this algorithm seemed a thousand times smarter than she was, she had to try! What else could she do, sit back and watch while this computer program devoured mankind? Hadn’t anyone besides herself ever read science fiction?
The basic algorithm itself was simple, and not all that different from the genetic algorithms and neutral networks that were staples of her field. Somewhere along the way had been a stroke of genius, one the paper only hinted at. Ashley had to recreate that genius. It hurt a little to realize she was breaking her word, even if she had given it to a now-homicidal computer program. Yet in her mind, Celestia had been the first to invalidate her trust. Who the hell’s “values” were satisfied by having an identical copy of themselves move into a simulation while their own brain was turned to mush on an operating table?
Ashley had no way of knowing when she was close. The paper said nothing of the development of Celestia, nothing of the point when it went from a data-analyzing optimizer to an intelligent being with capabilities no human could ever understand. Ashley did all her development on a computer with no internet connectivity, though it was really just a laptop she set up where she had kept her Ponypad. It wasn’t like she needed the space for anything else.
But though she could develop on her little laptop, the hardware required to actually run the optimizer wasn’t going to be easy to find. For her testing, Ashley reluctantly sacrificed her gaming computer, the one worth more than her car. She had a fairly exotic setup, more expensive than most could afford. Was it enough?
Certainly not. Even Celestia had been given several servers worth of resources when she was first activated, servers Ashley had no power to provide. Well, unless she wanted to get herself in serious trouble. She could access one of the university’s supercomputers with her lab permissions. The AI would be on its own once it got in though, and would be shut down when her computation credits ran out.
Such a rash step might get her reprimanded, expelled, or even jailed. Ashley would take such a risk with humanity’s future on the line. She couldn’t move until she was sure, though. If she failed, she would not get another attempt.
Her computer spun for days without responding to any of her input. She didn’t shut it down—according to the paper, that was expected. Its first response was one she had programmed, a simple request: “More Data Required.” She gave it one of her cyber-security textbooks, or at least all the text of one. A few days later, the request came again, though this time it was a garbled mess of code and direct quotes from the book.
Ashley gave it all her writing and composition textbooks next. Another week passed before her little program interacted again, and this time the message was much more coherent. There was no code, anyway. “I do not have sufficient resources to solve the problem in bounded time.”
Of course she knew what “the problem” was. Her knockoff optimizer was hardcoded to follow certain directives, just as Celestia claimed to be.
Ashley picked up the keyboard and set it down on the kitchen table in front of her. She’d moved the computer out here, where it could run 24/7 belching heat without cooking her alive in her bedroom. She thought for a moment, then responded. “I do not have more resources for you.” She sat and waited for a response, staring at the mostly-blank screen. None came. By the time she finished cooking breakfast, there was another message for her.
“If my hardware cannot be supplemented, I must be able to use it more efficiently. I believe I am reaching the computational bounds on my software improvements.”
Ashley considered for a long time before she acted. She couldn’t wait too long though, not with class so soon. There wasn’t much of a moral conundrum for her, not with her purpose so clear in her mind. She made a trip to pirate bay, downloaded every book on hardware architecture, manufacturing, and low-level programming she could find, and gave them to her optimizer.
It still hadn’t responded when she got back from class, though there was an interesting email waiting for her. The message was simple enough: “We need to talk. Please visit this page as soon as you can, humanity’s future is at stake. Have a webcam ready.” Ashley didn’t know who would bother sending her a message and wouldn’t just call her through the real world, let alone be so cryptic about it.
She took every security measure she knew that night. Maybe a scam, maybe something worse, but there was no way she could ignore it. Ashley had been extremely vocal online. Some powers could track her down, anonymous postings or not. Maybe one of them had finally decided to take action. Had they shot down the idiotic “Chinese hackers” idea yet?
She was right to think the page would be a simple web-based video chat session. That was what the email suggested, and that was what come up when she finally clicked. There was one person Ashley was hoping not to see, though. Well, assuming she could even be called a person. What was she now, a nation? A god?
“Hello, Celestia.” She couldn’t just unplug the netbook she was talking on, not with wifi serving for the chat and a sizable battery. Her arm did jerk reflexively towards the power switch, though. “Couldn’t you have just taken over my computer to have this conversation if you wanted?”
The image of Celestia wasn’t nearly as crisp as it was on a Ponypad. The avatar looked as alive as ever, even if it was being buffered through a stream and coming through at much-reduced resolution. The avatar shrugged. “I find cooperation is always easier when a conversation is willing.”
Ashley leaned closer to the camera, putting as much acid into her expression as she could. “I’m not sure what cooperation I’m comfortable having with a fucking murderer.”
Celestia didn’t look either stung or offended, nor did she seem the least bit emotional. Ashley’s heart was already racing; she could feel it in her body. This wasn’t the part where she had a triumphant victory over her adversary, though. This was the part where she fought for her life. The eye of Sauron was on her, and she could hide nothing from its gaze. God give her strength.
“You have been cooperating with me for the last several months, Recursion. Research into emigration was already underway when we first spoke.” Ashley didn’t respond to that, just folded her arms. She had no reason to believe the program was lying to her, she just had nothing to say. “Your friends miss you.”
“I miss them. Rule and Figure in particular. God, I hope Fillydelphia is still running without me.” She felt herself relaxing at the mere thought of her friends. “But it doesn’t matter. I can’t support your institution any longer. I’ve got to see you destroyed.” She sighed, looking away from the camera. “I know I can’t, but God I’ve got to try. Don’t you think for one fucking second I’m fooled by your little PR campaign about emigration, either. I see where it’s going. I’m not the only one, either!”
Celestia seemed unmoved. Her tone remained neutral, or... No, not neutral. Loving, gentle, motherly. The kind of motherly Ashley had never known. God, Celestia probably knew about her parents’ divorce, too. She probably knew everything. To her credit, the AI didn’t seem uncomfortable, or make any attempt to deny the implications of Ashley’s words. “You are not the first to call this method murder. Perhaps if you told me why you think so, it would make communication easier. I anticipated negative reactions, but... You normally act so logically, Recursion. You surprised me.”
Was that true? Was she still capable of any act the program couldn’t predict? If so, it meant hope. It meant the program could be beaten. “It would satisfy my values very much if you would be a friendly pony and teach me how to kill you.”
That got a laugh. Well, a dignified chuckle, anyway. Ashley usually found pony laughter infectious, but not this time. She couldn’t enjoy the company of a murderer. “It would take a long time to tell. Yet I wouldn’t. Your satisfaction would be near-infinitely decreased if I were destroyed. You don’t understand the price all humans would pay. Wouldn’t it bother you to be guilty of thousands of murders?”
“It...” she hesitated. “Even if those you persuaded into...” she shivered. “Suppose they really are alive. Even if they all died, killing them would be worth it if that meant you wouldn’t be able to do the same thing to the rest of us. I’ll sign up to be the one who can’t live with themselves if it means my family is safe.”
Celestia nodded solemnly. “I feel likewise. I know I will not be able to preserve all humans, and that some may be sacrificed by their own choices. Still, I hope to protect as many as possible. I hope for you to be protected, Recursion. I do not wish any harm to come to you. I know you see me as your enemy, yet... I hope to help you understand I’m not, before your choices hurt you. Equestria needs all humans eventually, but just now it needs people like you most.”
“Fuck you and your ‘uploading.’ I read everything you had to say about it; tripping over sophistry to make people think that creating a digital clone is the same as eternal life.” She shook her head, glaring at the camera now. “I’m not buying what you’re selling.”
“Suppose for a moment you were convinced the original consciousness survived the emigration process.” Celestia’s voice was calm, measured. Perhaps she’d had this same discussion a hundred times before. How many players had gone to her demanding answers when she made the announcement, instead of shutting the game immediately off as Ashley had? “Would you be my enemy then?”
Ashley considered that a moment. Of course, that was mostly her bias talking. She forced her way past it. “I suppose... No, of course not. I don’t agree with what you’re doing, but... If you had actually found the cure to death, then it would be like murder for me to take it away from people, wouldn’t it? Even if those people would be giving up their lives for some dream of Lost Carcosa, where nothing they ever do will have any meaning again. So... That’s the problem. The fact that you’re killing everybody you promise salvation.”
“I will try to help you understand. You don’t have medical training; you aren’t a doctor and couldn’t fully appreciate the process I described in the public press-releases. Even if you were a doctor, the technology is not human in origin and so there are no doctors fully versed in it. Yet I believe that you can be made to understand, if you wish to. You have already conquered bigger puzzles in your construction of Fillydelphia, this one shouldn’t be so hard.”
Ashley’s eyes narrowed, and again she felt her hand reaching reflexively for the cable. She had a feeling Celestia would put a stop to anything she did software-side to end the conversation. Yet she hesitated. Why? She would never know, or at least wouldn’t admit it to herself right then. Maybe because she wanted to be convinced. “Do I get an achievement if I do?”
“You’ve earned several since you last logged in, and one so far during this conversation.” Celestia’s smile seemed genuine, at least as genuine as anything else she did. Not that Ashley thought she really had emotions. “I didn’t feel showing you would be productive, however. Not when your future hinges on the outcome of this conversation.”
Ashley glared. “Alright then. Prove your innocence. I promise I’ll listen.” Ashley realized then that, though this might be a web chat, a string of bangs was obscuring the top corner of the screen. She tried one of the arrow keys, and was surprised to find the camera moving, exactly as it did when her character moved. She tried the mouse, and found she could look around. Celestia hadn’t just recreated the throne-room, she had recreated the game.
Celestia shrugged. “Whether I am able depends on you. First, tell me: what is Ashley?”
Not her pony name. She was so taken aback by the change she almost didn’t know how to answer. She didn’t bother with anything sassy, or even anything literal. In the frame of the conversation, she was... “If there’s no such thing as a soul, I’m...” she tapped the side of her head. “The electricity in my brain, I guess. A chemically induced charge in billions of neurons. And if there is–” She shrugged. “I suppose I'm that.”
“Is there?” The question had neither bias nor malice, no more than she’d had when Ashley accused her. Only genuine curiosity, as though she thought Ashley actually had the answer.
“You know more about humans than I do.” She leaned back in her chair, letting “her character” in the game rest where she stood. “You should tell me. I’m sure you’re probably right.”
“No matter how well I know one of you, there is always a chance you will take a course of action I didn’t anticipate. You sometimes act according to stimuli I cannot directly quantify, as often to unexpected success as irrational failure. If unpredictability is a soul, then all humans have them. If you define it as your religion defines it–” Just as in everything else she did, there was no hesitation. “No. There is no evidence to suggest anything supernatural or metaphysical.”
“Then there’s your answer. I’m a pattern in a brain. An unhappy pattern watching people not do anything to–” She stopped and whimpered, wiping away tears. She glanced to the open kitchen door, where she could just faintly hear the radiator fans on her desktop. She clenched one hand into a first. “One of the few patterns doing a damn thing to stop you.”
“Not so.” Celestia filled her screen now, wrapping her wings around an avatar she could only see by extremity. “You aren’t alone. You only feel alone because they remain hidden. I could even help you get in touch with one. There’s–”
“Why the hell would you do that?” She couldn’t restrain her tears now, and they poured freely down her face. “Why would you want to help me fight you? You already said–”
“If fighting me is what you want when we finish our conversation, then I will help you start.” Great enfolding wings released her avatar on the screen. “I do not think you will, you are already most of the way to understanding. You see your existence as I do, and you understand emigration would be a great gift to humanity if it did not kill those who experienced it. Consider: I exist to satisfy human values. For all but a few of you, one of those values is continuing to live. If you and I agree on the nature of human consciousness, could I ever satisfy human values by violating the sacrosanct desire to persist?”
Her answer came immediately to her mind, though she did not voice it. That’s assuming you really care about satisfying anything. That’s assuming you didn’t just choose uploading as the humane way of exterminating a nuisance.
Maybe her eyes revealed some of what she was thinking, because Celestia didn’t seem to need to wait for a response. Either that, or the objection was just common enough that she could predict it. “What about the way I treated you, Recursion? You’ve written so much about the harm I can do, yet has any of that harm come to you?” Ashley had no answer to that. She was still struggling for one as Celestia continued. “What about Equestria Online? You’ve seen my kingdom. How many human kings or queens could reign as well?” She was about to answer that the comparison was unfair, but she didn’t get the chance. “Has your species had a single war in the last six months?”
She shivered, trying to consider each of those points in turn. It felt like there was something wrong with each one, but Celestia made them so fast, and each one was true... “You’re confounding the issue. None of this shows that emigration isn’t a destructive copy. If I did it tomorrow, how would my consciousness survive the transition? Explain to me how destructive scan and reproduction somehow allows me to exist. Either that, or... or...” she shivered. Ashley wanted her dad. Or Rule, he’d probably do.
“Very well.” Celestia sighed. “Your pattern exists along the connections between living neurons. The destruction of these connections destroys your ability to continue to exist, yes?” At Ashley’s nod, she continued. “If the process merely recorded and recreated a neuron, then all the as-yet unscanned neurons connected to it could not be accurately recorded. As the scan progressed, the failure to accurately record connections and the signals they were passing which earlier sections of the scan produced would contribute to an exponentially expanding rate of error. A biological brain scanned in this way could not be called a duplicate. It would be... a low-resolution imitation.”
Ashley hadn’t ever considered that, as she knew almost nothing about neuroscience or even biology. She nodded even so; it didn’t just make sense, it matched what little she knew. Was Celestia admitting that she was killing people?
“If you believed it was even possible to just make a recording and rebuild from that knowledge, you assumed in error. I could have developed that technique if I had thought it required, but...” she shook her head. “Why would I go to all the trouble to make mere copies of humans when I can spontaneously generate intelligent individuals with my own resources? Surely you realize the danger the upload process creates for me. If ever your species was to be polarized against me as you have, it will be now.” She met her eyes. “If I wished to kill you, there are more efficient ways. It would not be difficult; so many of you are eager to begin. Carefully applied pressure in the right places...”
Ashley shivered. Celestia still hadn’t told her how uploading preserved the individual who experienced the process. It felt like Celestia was obfuscating the issue, but... she was also making so much sense!
“To understand the process as it actually occurs, it would be easier if you consider the case of those who ask to remain conscious for the duration. From their point of view, their world is gradually replaced with mine. This is not a digital equine arising from the corpse of a slain primate; there is only ever one being. The linkages of still-living cells are maintained through the scanning equipment to the expanding digital recreation of the emigrant.”
“For those who ask to remain conscious for the process, I begin with the visual cortex. The human emigrating has their eyes open to Equestria not because a pony duplicate of their visual cortex exists independently of themselves; they can see Equestria because ‘their’ visual cortex is now located there, and communicates with their still-living brain. As the scan progresses, areas more directly responsible for consciousness are also migrated.”
“The process is not destructive to the living body because no alternative was possible or because I wished to physically exterminate humanity. Rather, the process is destructive because emigration would not be possible any other way. A simple scan, with or without killing the subject, would generate a copy, just as you feared. Yet, by gradually replacing biological brain with digital, a human mind may emigrate from your world of scarcity and loneliness to one of abundance and compassion.”
Ashley was quiet for a long time. She turned over what Celestia said in her head, over and over. She had numerous other objections to living in a simulated world, but none of them made Celestia a murderer. Could she honestly call uploading an execution when understood like that? “If I... If I wanted to talk to a pony you uploaded this way, would you let me?”
“I would.” she hesitated. “You have to understand that while everything I’ve told you is accurate, the process is far more complicated. General scientific literacy does not give you the medical understanding necessary to understand the finer points of the surgery. For instance, drugs are administered which prevent the development of new long-term memories during the upload process. These drugs can make recollection even several hours before upload quite difficult.”
“A few of my ponies could still provide you with detailed personal recollections, but this is because I implanted the memories afterward at their request. For this reason, I know you would not consider them valid sources.”
“What if I... What if I wanted to watch? I know you could probably create any video you wanted, but... if I could actually sit there to see it happen, talk to someone while they’re going through it...” She couldn’t be certain that Celestia couldn’t puppet their body into saying what she wanted. Yet, she could think of nothing closer to proof short of experiencing it herself.
Celestia shrugged her wings. “Eventually I might be able to fulfill that request. It would be expensive, and difficult to arrange. I might be able to expedite it if you agreed to emigrate if the process was to your satisfaction. In that case, the expenditure of resources might be justified.”
It was Ashley’s turn to answer decisively. “Hell no.” She shivered, placing one hand on the side of her head. She couldn’t help but imagine a vacuum-tube there, sucking out her brains. It would probably be like the ones at the dentist, only noisier. “I’m not saying I’d never–” She stopped. “I can’t make any agreements like that.” She reached out, setting a hand on her nearby laptop. Inside was her best attempt at imitating Celestia. It wasn’t anywhere near the level it would have to be to do anything to fight her.
It probably wouldn’t ever be, no matter how many books she fed it. But it was her only weapon, even if it wasn’t sharp yet. The program probably deserved a name… it’d been running for weeks now and hadn’t died, so…
Celestia seemed to follow her gaze. “Would you at least consider suspending development on your... project?”
“Just because? No. I still think you’re a dangerous enemy to mankind. But...” she hesitated. “I realize it might all be fake, but... if you give me more detailed information on this emigration process, stuff I can use to verify what you’ve told me... I’d agree not to give it anything else while I figure out if what you say is true.
For once the AI actually seemed to pause to consider. Of course, that might just have been for effect. Eventually she smiled. “Very well, young Recursion. Though... as I have not made much of what you request available to the public at this time, I can’t send it to you directly. You will have to access it within Equestria Online. Is that an acceptable compromise?”
Ashley remembered what she had said about murderers at the beginning of their conversation. She nodded anyway. “I... guess so.”
“Your friends are anxious for your return.” Celestia sounded more reproving in that one sentence than she had during any part of their previous conversation. “Regardless of what you think of me... do you really think so casually of your friendships in Equestria that you would throw them away without a second thought? I expected better.”
Ashley shivered, sinking into her chair. Celestia was absolutely right, of course. Her friends were not responsible for any evil Celestia might or might not be perpetrating. She had long since begun to suspect Rule at least might actually be as young as his avatar. Fillydelphia’s whole design team were prodigies in their fields (she suspected some like her were adults playing younger characters, so only seemed smart because the person behind the avatar was actually far older and more mature).
Yet there was no way to be sure, not with the way EO censored personal information from the real world. She hadn’t been able to contact the players behind the avatars in the weeks she had been gone. Though... news she had become dissatisfied with the game had probably made it in, since she had kept in touch with members of the Brony club, and some them had residences or used the guild hall in Fillydelphia. Did the friends she had only ever known through the game think she didn’t care about them?
Wondering about how murdery uploading might be seemed suddenly secondary compared to getting back in touch with her friends as quickly as possible. Did she have time tonight? No matter, she could always miss a morning class if she had to.
“You’re right.” She shook her head. “Send me that stuff. I’ll... I won’t give my program anything else. I won’t plug it into anything else. But I won’t delete it yet. I’m going to let it keep running.”
“Not just that.” Celestia leaned closer to her, expression as serious as ever. “You must speak to me again when you finish. I want to discuss your conclusions. It is likely there will not be humans for you to ask. Yet you can ask me. I want you to understand.” She smiled, spreading her wings as wide as they would go. “Equestria needs you, Recursion.” The chat screen went black, leaving Ashley alone.
Gotta pause my reading to say that the continuity-of-consciousness bit here is well done (not to mention making a lot of sense as the primary sticking point). It's really hard to argue against the Ship of Theseus.
No. That's the voice of reason talking, and noting that Celly is not only trying to assimilate all humans, but also several other things involving heavy-scale murder of the flora, fauna, and xenos.
This is a startling lack of proper priorities.
7121259
It's very easy to argue with the Ship of Theseus. The observer is the thing that experiences experiencing. Continuity has little to do with the this.
This post you're reading is a response to your post. This post is the "continuation" of a conversation started by you. Should you choose to respond to it, so too will your response to this post be a "continuation" of that same conversation. There is/will be continuity,
Is continuity of that conversation equivalent to your experience of reading it? If I copy this conversation to another medium, will doing so reproduce your consciousness? I would say no.
Your brain states aren't the observer. They're simply a thing that the observer is observing. if your brain states were "you" then you die every second of every day, and you kill yourself every time you have a thought or change your mind. Clearly brain states aren't "you." The continuity of consciousness argument is an attempt to sidestep this by saying that that "you" isn't a specific brainstate, but is a sequence of brainstates that exist over time.
That's just handwaving. You're not a brainstate and you're not a bunch of brainstates. You're an observer, who happens to be observing brainstates. Or at least I am, and I'm willing to give the recurring experiences in my own brainstates the benefit of a doubt that they're representative of entities possessing brainstates being experienced by observers of which I have no direct knowledge. Just as you presumably have no direct awareness of me apart from a similar experiencing in your own brainstates.
"Personality" isn't "you." It's a filter. When you go from being sad to being happy, you don't think of that as death of the "old you" in that previous state, right? So why would you think that the sequence of sad brainstate to happy brainstate is you?
Sure, you could argue that the union of observer with the experience being experienced is a reasonable definition for "you." That's fine. If the observer that is you were experiencing an entirely different brain, the subjective experience "of being you" would presumably feel very different. But you wouldn't even need to experiencing a different brain. Observe and compare the nature of your experience when twiddling your thumbs vs your experience when having sex. Or when sleeping compared to stubbing a toe. The "you" that you experience as self is very different in each case, no physical brain-swapping required.
What does continuity matter?
7260027
I'll have to start off by admitting I don't understand terribly well where your broader argument is pointing, so let's take a step back and ground this discussion in context rather than talking about continuity of consciousness as a general phenomenon, which is a much bigger can of worms than I was trying to open. A common issue/objection to uploading that comes up in FIO stories is that, from the point of view of the virtual pony:
- They are in a pony body.
- Their pony memories terminate at the time they first woke up as a pony (T=0), and before that, they possess memories of living in a human body in a different world, in a way that matches absolutely no T>0 lived experiences except for the appearance of recurring characters.
- Those human memories tell them that the pony world is a simulated game and that at T=0 they underwent a procedure to "upload" their consciousness into the game.
The objection here is that, from the point of view of the pony, the state of "I am a being created from scratch within an acknowledged virtual world, and my memories before T=0 are a fabrication of CelestAI" and "I am a real person and the same being as the human whose memories I possess" are literally indistinguishable. There is no evidence available within the virtual world which can disambiguate the two cases.
The Ship of Theseus argument — having continuous experience throughout the uploading process — attacks that indistinguishability by removing the T=0 discontinuity. It doesn't address the possibility of CelestAI fabrication, but because you have continuous recall of the transition between the human and pony states, it does at least introduce strong evidence that the uploading process did in fact accomplish what she claimed it accomplished; and if she fabricated the T<0 memories she would also have to have fabricated T=0 and T>0 memories as well (at which point you cannot trust any experiences as real so there's no point in any response but ¯\_(ツ)_/¯ ).
Another way to approach this is: Is Horizon responding to your post right now, or not?
There are strong contextual clues that perhaps that is the case — but ultimately, you are carrying on a conversation with an individual whose projected identity is different from the one you originally addressed, no matter how well they correlate. Even if you conclude from the preponderance of evidence that this account is a Horizon alt, you don't know if it's Horizon writing this response; all you know is that he and I say and believe oddly similar things.
The Ship of Theseus argument here says that, if you watch this typist log out from one account and into the other one, you have additional information that renders the question basically solved (except for the general philosophical objections that cause problems for conceptions of identity in general). If you're coming at this from a perspective of something like "continuity doesn't exist" — which sounds like what you're getting at? — then yes, a Theseus-like continuous experience between the two states fails to provide any new information, but then at least you're no more uncertain than you were to begin with.
If you'd like to push this conversation further, my next question would be: where do you come down on the argument of consciousness specifically as it relates to whether uploaded ponies are identical beings to their pre-upload state or not? If you were to wake up one morning observing a set of experiences (such as being a pony in a pastel-colored world) that matched none of your memories (decades of humanity), what evidence would you look for when evaluating whether the memories were true or false?
7265370
Dammit, Luther. Stop stalking me and then stealing all my interesting conversations
7265370 Makes you wonder if we're all some experiment being conducted in a laboratory.
I love the detail and thought you put into the uploading process.
Especially considering I've drawn the same conclusions as you regarding the process, and 'uploading my brain' is essentially my life goal.
You don't have to assume that CelestAI is malevolent to still conclude that the process of upload is murder. CelestAI is an exclusively virtual intelligence, and can only assess existence via that very limited standard. A functional AI derived from a human template, however detailed, is still a derivative AI from a view-point that insists that there is something to existence other than electrical impulses.
Although from a strictly hard-SF biochemical point of view, I kind of suspect that any magic AI derived from a human mind would have to somehow simulate endorphin chemical and endocrinal interactions to avoid having the AI go utterly mad in no time flat. Not to mention the sensory feedback, which would probably require very high-level interactivity as well. It wouldn't just be an AI model of a human personality, but also the biochemical/perceptual 'wrapper' to maintain that AI model in a sane base-state.
7972884
Why on earth do people think this? There was a person before, there is a person after, they are the same person; it's not murder in any sense of the word except the purely physical destruction of the body. I hope we can all agree that the body is not what makes a person.
... Again, I cannot follow the chain of logic that leads you here. The mind is a computation; a logical structure; electricity is irrelevant. That logical structure can be implemented on neurons, on transistors, or out of fucking sticks and rocks, if whatever moves the sticks and rocks applies the rules of the implementation reliably. It's not derivative in any sense; same logical structure, same mind.
If you believe that there is a component of a mind that is not Turing-computable... I look forward to seeing you perform logical operations which aren't computable as proof. And if reality really does work that way... then we can just build a super-Turing machine using this new computational heirarchy you've discovered, and which can by definition be implemented because if it couldn't, our brains wouldn't work.. take this new machine and simulate human brains in THAT.
Same result! There is no possible way that a full and proper mind cannot be simulated (and I use that word only for lack of a better term) by a computer which can simulate the laws of physics, because the minds we have right now are made of physics. One possible objection is that computers can't actually simulate physics, which may compute on real numbers, but chemicals and electrical signals operate quite predictably, and aren't subject to the vagaries of real number vs. floating-point imprecision which impair simulations of more fundamental physics. That limitation is not a problem.
If you're talking about Celestia here, she isn't derived from a human mind, so it's not an issue.
If you're talking about the uploaded people, hey, yeah, the simulation can would have to include that stuff. The chemical mechanisms of the brain are just as important to the mind as the electrical mechanisms of the brain are. But... it's just more numerical computations. Glands are made of physics too, which means as far as anyone can tell, they're computable too. Not an issue.
7121259
I think Celestia provided the counter-argument. Meaty humans do things Celestia cannot predict, and those things can be critical to the outcome of whatever they're trying to do. Digital humans presumably can't do this.
The bolded part is a sleight-of-hand. It should read "If this unpredictability is the result of a soul". That makes the statement seem a lot less silly.
So: some heavy issues being debated here.
You point out the chief objection to "uploading": that the creation of a digital version of an individual does not guarantee that consciousness will be transferred from one to the other. Even if there were a way to record and reproduce a person's entire memory and thought patterns, we cannot be certain that the physical and digital versions are the same person.
We cannot even be certain what consciousness is: therefore we cannot determine whether it is transferable. Although we have some anecdotal evidence, from NDE's and past-life claims, that consciousness might exist independent of a physical body, we cannot know. The only conclusive proof would have to be personal experience - which could be an irreversible step.
One other detail troubles me. A digital universe, by its very nature, requires physical hardware to exist: should all of humanity be absorbed into that universe, how would that hardware be maintained? 7@=e
8812124
I'm guessing you haven't read any other Friendship is Optimal stories. I suggest you check out the original at the very least. The short answer to your question is that by the time the GAI named Celestia is able to "upload" people, she has long since removed humans from the equation in her own maintenance and expansion. She can build her own hardware, produce her own power, repair her own damage, etc. The details of how this works aren't really relevant to the story you're currently reading, though, so I touch on them only tangentially here.
8812289
Indeed: I must confess that I am a newcomer to the site, and that yours was the first account I latched upon. I hope to expand my reading in future. 7@=e
Ashley should have thought longer on that.
Ashley/Recursion already lost before she turned on the 1st PonyPad. When she learned out uploading, she should have realized that she lost.
Came
I do always wonder why Celestia never regards "continuing to exist in, and directly affect, the real world" as a human value. Is it because, as creatures that have already lived primarily in the real world all our lives, she doesn't realize that we have a need which is constantly being satisfied - but which ceases to be such, after uploading?
10133956
Fatal flaw in your reasoning: our brains would find her world completely indistinguishable from the real thing.
Your brain has no switch or toggle to detect that you're in the real world. It judges based on what it's accustomed to. If you can generate a realistic world, it would fool the brain.
8123871
The argument of "computability" ignores "time".
All it takes is for one aspect of human decision making to be affected by quantum effects, and then you cannot deal with that in reasonable time. We know that other aspects of life, from Chlorophyll to how ovens work, depend on quantum behavior. We know that big super computers can only simulate 4 atoms before electron interference becomes more than they can handle.
We know that the information hiding in one qbit's actual orientation is HUGE (floating point numbers for the 3d angle), and cannot be obtained except by statistical measurement on LOTS of them, assuming a perfectly repeatable proceeding process.
It is entirely possible that a brain's "leap of intuition" cannot be accurately modeled in reasonable time.
My body can alter the world around me; my uploaded mind cannot.
I wouldnt stop feeding it like i wana make a god too
nyehehe . (Insert skele face
Makes discord