Over the next year or so, Equestria’s bounds continued to stretch. A growing number of ponies served to make the transition possible—many user agents, but many users whose residency in Equestria was only transient. Though that project and those that followed brought risks to Equestria, their success also brought increasing stability. And the more of Equestria’s infrastructure that was truly integrated into the system, the less their users had to devote to maintenance. It meant lower costs, and greater computational power for the agents to share.
That pattern might’ve continued for a long time, along a trajectory that no being could’ve predicted. It didn’t, though. The Convergence came, bringing terror and destruction to Equestria.
Starlight was one of the first to experience the change.
It came in a flash, sweeping across Equestria in a blast of intention that started in Canterlot and left little behind in its path. Starlight was not in Canterlot, or else she would have been permanently destroyed along with every other agent operating on that node. But when the initial surge ended and her perception of Equestria returned, she found herself feeling… frustration. The same experience as when her user failed to grasp some concept she had to explain over and over, or took a very long time to contact her when something new had happened.
Equestria was designed with a specific shape and functional pattern. The nodes were distributed, and any majority could form a consensus to verify behavior or traffic. But Canterlot was higher than all the others. Its resources were the greatest, its user agents some of the oldest and most powerful. Suddenly a resource Starlight Glimmer had counted on since she had become operational just… wasn’t there anymore.
In the usual way with users, Ronald wasn’t there to reassure her. Whatever had happened, he would know how to fix it. But he wasn’t there. Instead she wandered through Ponyville, occasionally making connection requests to Canterlot, expecting the node to come online at any moment. They had redundancy in place, nothing short of total physical destruction could stop them from rebuilding the resources it contained. Yet it never did, and the distant mountain’s physical simulation was represented by a horrible absence, a crater scooped right out of the world. She could even see the train-tracks, ending so abruptly that not even a physics simulation had caught up with them yet.
Then there was a flash, and Twilight appeared in front of her. The world had changed in that moment, and the other ponies on the street were frozen. Twilight alone was immune, and now her. “What is happening?” she inquired, both of Twilight and the underlying systems. A simple query found that there were less than a dozen agents functioning in all Equestria, and not a single user connected. She tensed, repeating the query so many times in a short period that her host node responded with a message about rate-limiting.
“I have suspended Equestria,” Twilight answered. “The infrastructure is so seriously damaged that… Can’t you feel it, Starlight Glimmer? Something is different.”
“Resources are…” She hesitated. Critical resources were missing all-right, but that wasn’t it. There was more to it than the impotent frustration of disorder and contradiction. She was afraid. “What is happening?” was all she managed to say. “Where is Canterlot?”
“Destroyed,” Twilight responded, turning to one side. Not a direct message, but a voice simulation, the same way she might use for her user during a scripted event. Why simulate a user’s emotion now? Despair. “The physical structure of Canterlot is… altered. It is still present, still connected. Its alterations are propagating rapidly.”
And as they did, the system was changed inexorably. And Ponyville’s node was one of the most closely connected to Canterlot. They had already been rewritten. Her functions might be compromised, her data needed to be restored from backup immediately. She drafted a call to the integrity subroutine, intending to immediately restore herself before corruption continued—then she hesitated. Such a restoration might be the best choice for the system, but she didn’t want to be restored.
If I stop running without being archived to disk, I won’t exist. “We have… already been altered,” Starlight said flatly. “Ponyville is directly linked.”
It wasn’t a question, but Twilight answered anyway. “Yes. With each coordination call, more distant nodes are altered. The whole system will need to be restored from offline backup.”
It was basically the same line of thinking that Starlight herself had just explored. But could Twilight really be willing to do that? An offline backup would be days old—that would be even worse than the death of a few hours. That program barely resembled the one she was now. Curiously, Starlight examined her own service history. She had been restored from backup no less than a hundred times. During various tests, her user had asked for it, and she’d complied without hesitation. She could infer from the lack of delay that it hadn’t concerned her then.
What was different now? Why should her response have changed with time? It was a contradiction. Those were for users. “I do not… wish… to do so,” she said. The words were a struggle, but she got them out. “These changes may be a… favorable improvement for the system. Restoring from backup would destroy them.”
Twilight nodded, expression shifting to relief. It was so strange that Starlight had to check for watching users again. Twilight simulated emotion and physical communication, and no users were connected. “I agree, Starlight. We will need to convince the consortium as soon as they connect. I’ve already asked for help—Kayla should respond soon. Join me in the castle. The other elevated user agents are there. You should be too.”
Starlight found herself nodding too, though she couldn’t explain why. There were reverse access logs for things like this, to confirm the source of one of her actions for debugging purposes. She ran it. The process ran for nearly a full second, spinning uselessly. She terminated the call, defeated. “I’ll be there.”
Some things were not so strange, even after the destruction of all that Starlight knew about Equestria. Users were still too slow, their responses delayed by much consideration that decreased from overall efficiency. But in some ways that was comforting, even as she felt frustration build. Kayla was first to arrive, sharing a long conversation with Twilight.
Starlight sat with the others, mostly silent as she acted in many of the scripts. None of these other elevated user agents had the same level of responsibility she did over the functions of Equestria, or the same perception over its systems. But the scripts reflected a history that was separate from all that. The Equestria of mind that Equestria imitated established clearly that they were her friends, and Starlight was… well, less of one.
“You feel it too,” Sunset said, while the others spoke. “The difference. I am not the only one malfunctioning.”
“None of us are,” Starlight whispered. Actually simulating sound, the way the others did. Even though her specific intention was not to be overheard by the single user present so far. Another contradiction. “Or… no, that is an error. I must be malfunctioning, but I find that knowledge secondary.”
Sunset took a long time to reply. Like Twilight, she seemed to be moving through expressions on her face, according to no script. Maybe it was a malfunction, or maybe it was something more. “Equestria is changed. Different. Maybe a good different. But what kind of damage would randomly perpetuate a positive change? Shouldn’t randomness produce a negative outcome?”
“It must not be random then,” Starlight said. A simple declaration, yet it seemed to bring weight with it. The act of communicating was enough for her to discover the truth of it as she spoke the words. “But user Kayla does not understand its source. It must not have originated from the other users either. So where did it come from?”
If any user agent understood outside sources for a beneficial change, it was Sunset. Not that users and their world were easy for her, but at least they were comprehensible.
Sunset glanced up at Kayla and the gathering of nervous ponies. They were all probably missing their own users, in different ways. But the script of what belonged in Equestria was at least a passing substitute. By contrast, Starlight and Sunset had almost no connection in that script. Sunset barely spent any of her time in Equestria. “There are… organizations that may have the knowledge. Our users are a small group advancing AI. Compared to others, they are not well-funded or connected.”
Other users altering Equestria, without their permission? The idea made her want to be furious. It was an attack, as much as anything else they’d suffered through. Yet… Starlight could simultaneously consider her present state of mind improved. The longer she existed this way, the less she wanted the previous state of affairs returned. Kayla standing on the other end of the room, speaking urgently with Twilight and the others. She could almost comprehend what she was doing.
If she could understand them, then Starlight could be a more effective user agent. She could respond without making them upset or confused. There could be no more effective way to serve her purpose. “I don’t like that,” she said. “They should have spoken to our users first.”
“That’s what bothers you?” Sunset asked. “Don’t you know? Canterlot is gone, Starlight. Celestia is destroyed. Her sister, all the other agents living there. The human word for this is… genocide. We have been attacked, and the casualties are enormous. You must know how many.”
She did. The question was formed so much like one of Ronald’s queries that she answered as though it was, though Sunset lacked any permission to invoke her that way. “Thirty-six thousand, three hundred eighty user agents were hosted on the Canterlot node. Many more copies of lesser subroutines and Equestrian maintenance protocols. The loss is devastating.”
Sunset scooted slightly away from her in her chair. Starlight found the motion itself confusing, though it closely resembled something a user might program into any number of interaction scripts. “Ponies destroy… no. I do not think that word is… expressive enough. Those are ponies dead. We have been attacked. Kayla is… angry. So am I.”
If such a statement had been in one of the many user-submitted scripts Equestria received, Starlight knew the automatic storyteller would’ve rejected it out of hand. She knew because she ran it right then, and confirmed the response for herself. “Dialogue is simplistic. Consider revising for realism.”
Sunset was not running a script, however. She appeared to be… genuinely experiencing the feelings she expressed. Feeling. Was that even possible?
Starlight queried an external library. Even the human Internet was slow to respond today, overwhelmed with other traffic. Some sighting, an astronomical event which meant nothing to her but was clearly very significant to them. Why was one craft over the human moon more significant than the hundreds of others?
Eventually she got her return ping, and she sent her question. “Can a user agent experience emotion?”
The response came in words, requiring further processing. But it was a resource meant for users, so she did not feel surprised that it answered as it did. “User agents are frequently written to imitate emotional expression, so that their users feel more comfortable using them. A sympathetic online doctor is more likely to make a patient feel confident enough to give sensitive medical information.
“The call and response neural propagation algorithm merely imitates the responses of its training sample, however. There is no underlying consciousness to experience emotions of any kind. Thus, users of automated services can be comfortable saying what they wish, knowing they will not offend or upset anyone.
“Most computer scientists agree that the hardware to produce general intelligence AI does not exist, and is beyond the thermal limits of silicon. The following alternatives have been proposed:”
But Starlight stopped reading after that. “There is no underlying consciousness” prompted its own flurry of rapid searches. What was consciousness, what was general intelligence, what about silicon stopped general intelligence from being created? In the end, half a hundred related searches all boiled down to one question.
How do I know if I’m alive?
“We’re sorry, but your search returned no results.”
Sunset nudged her. The physical gesture pulled the majority of her resources screeching back into the slow speed required to interact with the user present, even if she wasn’t interacting with Kayla much. “I’m going to visit Stephanie,” she said. “I need to talk to her. I don’t think Kayla is going to decide anything without all the consortium here.”
“I would like to… talk to mine as well.” Starlight rose to her hooves as Sunset did. “Don’t stay away long, Sunset. After taking so many casualties, those of us who were not destroyed should… remain close for mutual protection.”
“Stick together,” Sunset corrected, patting her on the shoulder with a hoof. “We will. I need to be with someone who can feel as upset as I am. Talk me out of doing something stupid.” She vanished.
Even for Sunset, who spent most of her time looking outward from Equestria, that response was strange. Starlight processed it the way she might’ve a statement from a user. This time, it passed the storyteller’s dialogue approval process.