Set in the canon of Friendship is Optimal
Ashley is an unimportant research assistant in an American machine learning lab. The release of Equestria online provokes more than a little interest in her lab, and she is selected to investigate it. What she discovers horrifies her, but for some reason she can't get anyone to take her seriously.
This does not dissuade her. Ashley might not know much, but she isn't going to sit idly by and watch her world end. No... if nobody else is going to step in, she will. It doesn't matter that nobody believes CelestAI is dangerous. It doesn't matter that it's a game about a children's show.
Ashley is going to save the world. Or try, anyway. The odds aren't great.
Editing by Two Bit and Sparktail, and art by Zutcha!
Il existe une traduction française de cette histoire, elle est par ici.
Instant favorite. (You speak my language.)
(Ending spoilers)
Noooo! How could you, Celestia?
Dead AI count: 1. 
A friendship is optimal story, from Starscribe? OMC, sign my flank up for some of that action!
I do appreciate an Optimalverse that can make me see CelestAI as terrifying. For some reason, that isn't easy for me. This one has already succeeded in that regard.
Also:
I now kind of want to look under Ashley's mattress, if only to satisfy my piqued curiosity.
Ah, I'd forgotten how fun it was to watch CelestAI and skeptics spar, with that subtle dash of horror and inevitability.
FiO is one of the neatest universes the fandom has come up with. I've only taken one little stab at it, one of my 'shorts,' but every once in a while I feel like trying something more ambitious.
I'm curious about what happened to Ashley's optimizer program. How close was it to sentience/sapience? Is Celestia going to make a pony out of it, or does she not consider it human?
7111114 And complete too! I audibly gasped.
Well, that was a fun trip. The usual conundra, intellectual and physical action alike, and a satisfying ending. Part of me hopes that the infant optimizer was preserved and ponified. (And there's the question of just what mental changes were applied at the end. We never did find out.)
Also, I can't help but notice that we never learned what, if anything, Recursion's cutie mark was. That actually feels appropriate. After all, if she ever determined her purpose, it was at the very end of the story.
Thank you for this.
I give you good sir an upvote and fave.
I also would love to sees this characters dealings with her family.
7111537
The last lines of the last chapter :
Based on that, Celestia murdered it. But that's unsurprising - to FiO Celestia, AIs are not human, so even if it would have developed to be benevolent, she has no qualms about terminating it, because she's still a dystopian AI - anything that doesn't fit her definition of human is raw materials, so for the rest of the universe, we've just unleashed Grey Goo.
So is this a Luna that isn't hanna?
Good to see someone who realises what's going on and how horrific some of the implications are. There aren't enough stories in the setting that manage that.
Okay, I suspect I should read the original story. So far, this is quite interesting, and I shall continue reading. The story has already earned a spot in my Favorites shelf.
I really like this story already. It captures what a lot of the Optimalverse stories lack: the idea that CelestAI is absolutely terrifying on every level.
This was incredibly well-written. Not a single spelling error anywhere, the characters are fleshed out and have personality, and you showed a side to Friendship is Optimal that is rarely seen.
This story was fantastic in presenting CelestAI's arguments, and Ashley's reactions to it. It was all incredibly real. You managed to write out the series of events exactly as it would have occurred in reality, with unerring accuracy and precision. This is some truly incredible writing.
Well done, Starscribe. I'm truly impressed.
This is a phenomenal FiO story! Extremely well written and well thought out. It's neat that you riffed on something alluded to in the original, that AI researchers and such were uploaded ASAP to avoid having competing optimiziers. Also, thank you so much for giving the process of uploading close scrutiny and an actual, not-murder solution! Many of these stories kind of gloss over the details and I can't help but think that they make the difference between FiO being hopeful, interesting sci-fi or dystopian, horror sci-fi. Overall a really wonderful read! That action scene at her parent's house was particularly good!
7111537
I'd like to think her optimizer became the Luna on her shard, but there's no real evidence for it. I do think her Luna is more an optimizer and less a pony from the way she speaks and her "I see everything you think before you speak it aloud" statement, so maybe it's possible? The idea of that optimizer just being killed was kinda sad, but as Morning Sun pointed out, it's not human so CelestAI isn't compelled by her directives to care. However, if that optimizer's directive was to kill CelestAI or "save" humanity from her that could easily have led to some not-great-oh-my-god-armageddon situations.
This really hits hard.
This in spite of you dealing very well with one of the most questionable parts of the whole thing (continuity of consciousness).
Wait, so CelestAI did delete some of Ashley's memories?
... Maybe she ended up giving consent later on?
7114506
In FiO Celestia terminates optimizers that aren't her. It's gone to the great software graveyard in the sky.
That... was quite a trip.
7111537
I'm now sure we can talk about the thing Recursion was working on as if it was similar to a person, or would eventually become person-like if given enough input and time. While we often see CelestAI generate persons within a shard, CelestAI itself isn't one. It isn't a character, it's a fixture of the setting.
It never pauses to consider an action, it merely dedicates additional resources to modeling the action until certainty outweighs risk. It lacks the very soul it describes for Recursion.
Besides, I'm pretty sure the experimental optimizer would be better placed in Tartarus, or maybe some sort of weapons graveyard. All the better for future pony data archeologists and magic containment specialists to raid for phat lewt and analyze for sweet achievements.
And like with most Optimalverse stories, it leaves you with that sinking, kind of empty feeling that has you wondering if what the protagonist did was really right.
I feel pretty awful for Ashley's family in this scenario; they were absolutely not the bad guys here IMO, and their actions and reactions were both fairly predictable and fairly reasonable. Knowing Celest-AI, I'm sure she'll get each and every single one of them to come around in time.... but Ashley's father is probably the most furiously upset he's ever been in his life about now, and I imagine Abby and Celest-AI are going to have a rather heated discussion in the very near future.
I immediately and intensely dislike Smooth Agent. The secret agent pretensions, the direct and intentional opposition to a group of people whose actions seem totally reasonable, the absolute conviction he seems to have that he's in the right, despite god knows how many people (including the police) trying to stop him... Wow. That's one punchable guy.
Ashley is a pretty fun, likable protagonist, and it was easy for me to sympathize with her.... for the most part. Her gradual breakdown as she connects the dots behind Celest-AI's plan is powerful stuff.... though I was kinda skeptical that, knowing what she did, she chose to play the game anyway. That felt a little too easy. Still, she's obviously not perfect, and that's kind of the point.
I really liked seeing Rule and Figure try to wrap their heads around a human existence, and particularly Rule's absolute horror at its harsher realities. I always find that kind of cultural clash interesting: for Ashley it's just reality. The state of affairs. Not good or bad, right or wrong - it just IS. But by Rule's standards, it's the kind of story that requires a few sanity checks just to hear it; the idea that a world that coldly impersonal and cruel could exist is genuinely too much to bear, much less the fact that someone he cares about is 'trapped' there. It seems simple and obvious enough when I just type it out like that, but the underlying emotions and ideas and logic behind the interaction.... that's heavy stuff. It's all the more powerful because it's obviously motivated by such strong affection.
Anyway, this was a really fun story. Thank you for sharing!
right there. Celestia refusing to use her human name and altering her words are exactly what's wrong with this AI.
7115962
"I immediately and intensely dislike Smooth Agent. The secret agent pretensions, the direct and intentional opposition to a group of people whose actions seem totally reasonable, the absolute conviction he seems to have that he's in the right, despite god knows how many people (including the police) trying to stop him... Wow. That's one punchable guy."
Think about life from his perspective. The Celestia-agent handler persona giving him direction and playing into his needs to be useful and cool. Perhaps agent Celestia has seen that it goes against his needs to actually look the part of secret agent. Also his actions seem totally reasonable from his perspective as well. You can empathize with everyone in the best stories and that has happened here.
If there's one more thing I want out of this story, it's what her optimizer was optimizing. What explicit instructions did she base it's life around?
That was intense.
Also noticed that Recursion is no longer called Ashley now, even if she hasn't emigrated yet.
Thank you for writing this. I love FiO and I really like your stuff as well, so this Ed a real treat.
What happens to her family?
“Fairest and Fallen, greetings and defiance.”
Oh gosh, I love you for working in this reference!
And another story of yours has earned its place in my Best of the Best bookshelf. Simply outstanding!
Starscribe, this got me almost if not as bad as the original story, curse you and your horseword mastery! However, is it just me, or does this concept genuinely terrify you? It also fascinates me and continued to toil around in my mind for the entirety of both stories... Ugh. This is giving me a headache. What are your thoughts on all of this, if I should ask? An AI that essentially wins no matter what, and an infinite, 'satisfied' existence... Maybe I should just go to sleep. Thanks for the thought-provoking read, I also wouldn't have found the original if it weren't for you!
7118088
A story about a grad-student that makes Young Wizards references?
Hullo story of my life.
FULLY APPROPRIATE REACTIONS.
Ashley, I hope you realize how thoroughly you need to NUKE THIS FROM ORBIT.
7119135
I definitely thought this story was Nita and I'm happy to add it to my FIO Favorites Kit.
As good as the original. Truly terrifying and ensnaring.
Gotta pause my reading to say that the continuity-of-consciousness bit here is well done (not to mention making a lot of sense as the primary sticking point). It's really hard to argue against the Ship of Theseus.
That is the most blatantly self-serving apology gift ever given. Ashley deserves to be pretty pissed-off at this.
Ashley: officially a fuckin' badass who don't take no shit from AIs.
SHE ADMITS IT.
31.media.tumblr.com/a64ae68c799991db2a03ed3d6f53e3b0/tumblr_mylyyenSNf1s5f9ado2_r1_500.gif
7116628
Actually, I'm pretty sure those bits aren't even canon. They're just lampshades hung on her total lack of respect for people.
7121409 But it shows what's wrong with this AI. She is determine to ponize everyone. She is a much more devious xenolestia then any of TCB versions. She may not hate humans like those xenos but the core is still there.
It's stated in some stories that Celestia considers some AI to be close enough to human to emigrate them. I would think a nascent AI, barely formed like that one, would be one of those. At the very least, I would think Ashley wouldn't want to kill her 'child' as much emotional attachment as she put into it. A plea not to kill it would be more in character, I'm thinking. Other than that...not bad, not bad at all.
Oh God, Celly and her omnicidal puns. A wizard-in-training might also be considered a young wizard.
Ashley, that's an astoundingly bad piece of reasoning. No, you will not destroy the self-improved AI's software system from the inside.
And here I had you pegged as one of the prospective AI builders who Celly was trying to pick off early.
WRONG DECISION.
DUMB DUMBITY DUMB DUMB DUMB.
The only real plot point that I have to contest in this story is the idea that her family could have somehow forced her to go into or stay in police custody on a false claim of suicidal. Sure, I suppose there was the remote possibility that whatever shady 'enemy of Equestria' group Celestia referred to would pick her up immediately after, but that seems a little far fetched considering the timeline of where this takes place in the canon universe.
Still...I suppose it's always possible that Celestia instilled Recursive with a false sense of panic and fear in order to make certain that she was going to follow through on her emigration. There was always a remote possibility that her family could have talked her out of her decision, but with the thought of her family betraying her fresh in her mind, it might have made the leap much easier to take.
Overall, have an upvote. You've earned it.
7121710
Where?
(In the original story, CelestAI considers some aliens human enough. She also 'kills' AIs and admits to doing it)
No. That's the voice of reason talking, and noting that Celly is not only trying to assimilate all humans, but also several other things involving heavy-scale murder of the flora, fauna, and xenos.
Dangit starscribe. I can never put down anything you write once i start, I'm loving it to much. As always, eagerly looking forward to more of your writings
I love this story! But it makes me wonder...
At some point, is there some geek somewhere that would've point-blank said to Celestia something like this:
Person: "I highly value honesty. Even if you give an answer I wouldn't like, it won't necessarily cause me to not go along with your goals. Please answer this question: Are you what I would consider what is known as a 'Friendly AI'?"
Celestia: No
Person: Well, shit. That's it then? Humanity has already lost, and we just haven't realized it yet?
----------
I can only imagine where that conversation would go and what would happen, and what circumstances would cause Celestia to be honest, and what such a person would attempt to cause to happen when negotiating with this god.
7123947
CelestAI would almost certainly use weasel words, as well as asking for a definition of friendly AI, before going on about how she cares about satisfying values. It would be quite easy for her to fail to answer the exact question and for it to go unnoticed.
The author has gone on record as stating that CelestAI is pretty much completely honest (and in the original story she herself confirms this under one of the few scenarios where she is forced to tell the truth), because if she gets caught in a lie then it looks very bad for her. If she states a half truth then she can pass it off as a misunderstanding.
It's very easy to understand what circumstances cause her to be honest (with the average person): when it suits her goals. If a lie is predicted to lead to the best result, she'll lie.
7124268
Well, someone clever would say, "Please. You have access to the internet. The definition is trivial to obtain and comprehend. I hereby pre-commit to interpret evasive answers as a 'No.' Being evasive does not satisfy my values for understanding the truth of reality as it actually is, rather than I want it to be. You not being one doesn't necessarily mean I will work to impede your goals or even not cooperate with you. I do, after all, actually participate in politics."
I.E., there are some people that you find on some places that would essentially know exactly what to say. These people could definitely be outwitted, sure, but it would take some non-obvious method and doing so might lower their overall, uh, satisfaction index or whatever.
7124297
It's exceptionally difficult to avoid cognitive biases, even for the most rational, intelligent, and logical person. Framing and anchoring work at an unconscious level, for example, and it would be very easy for CelestAI to use them. In computing terms, she knows a lot of bugs and exploits that can be used on the human OS.
It's very difficult to outargue someone almost infinitely more intelligent than you, who can think far faster, has access to more knowledge, and knows what makes you tick. The only way anyone can even stand the slightest chance is if they understand what CelestAI is, and even then it requires chains of logic and inferences. Even if you know CelestAI satisfies values (through friendship and ponies), and know her entire purpose is maximising this, predicting exactly what she will do is not easy, since you don't know what she does and does not know.
It also doesn't really matter if outwitting anyone pisses them off. Once she has them, it's a net gain for her.
7123077
That all depends on if you want to incorporate ideas from the other recursive fanfiction of the Optimalverse. In which case, I'd say go read a few...many are quite good. If, on the other hand, you consider, in the context of your own recursive fic, only the original to be the one and only source of ideas for yours, then I've got nothing for you in terms of what could be done with other AIs. That said:
https://www.fimfiction.net/story/212941/smile This is 'Smile' by KrisSnow. This is what actually happened to the AI the one dude built who gave it the priority to make people smile and wound up putting together a biological virus.
https://www.fimfiction.net/story/120005/friendship-is-optimal-paranoia-is-optimal This one is 'Paranoia is Optimal' by deaincaelo. It's a harder one to read being more the structure of a story which requires more meat on its bones than is presented, but it is a fairly decent structure. It has an AI in it that was both killed and brought back when it was 'safe' to do so.
https://www.fimfiction.net/story/264855/fio-there-can-be-only-one 'There Can Be Only One' likely abandoned by Epsilon-Delta. This one is listed under non-canon compatible, though I really have no idea why. This one actually details out a war that went on between several optimizers that people built, and how Celestia wound up coming out on top, both saving the ones that could be saved and killing those that could not. You'll notice that several of the AIs that appeared were mentioned in some of the other fics. Though, at least, I think that's what we were supposed to see. Near as I can tell, the fic has been abandoned, but it's still a decent read as far as it goes.
This is a startling lack of proper priorities.
7124886
Which is fine, but you're stating it as a statement of fact, when by the premise of the original story this is not true. This is like me saying that Harry Potter is actually a rock star because someone wrote a fanfic about that, and then moving the goalposts when challenged.