Two nerds experiment with AI, but as it advances they find it increasingly difficult to control.
Keeping up with the latest magical research coming out of Canterlot Academy, Sunburst finds something interesting and decides to try it himself. After Starlight sees what it can do, she comes up with ever more ambitious ideas for how to improve it.
Sunburst made a note in the margin of the page he had already covered in a mix of equations and arcane glyphs: HUGE FAILURE.
He leaned back in his seat with a heavy sigh while he levitated his glasses off his muzzle and rubbed his face. The three-by-three grid he'd been staring at since last evening was burned into his retinas, continuing to taunt him from behind his eyelids. At least that banging sound had stopped...
"There you are!"
Sunburst fell out of his chair with a shriek and quickly scrambled to get back up, squinting at the blurry pony who'd snuck up on him. His glasses, wreathed in turquoise magic, found their way back onto his face and she came into focus. "Oh, Starlight. You surprised me."
"Sorry about that," she said, adding, "Are you okay?" The strained tone of concern in her voice and the way she hid her mouth behind her hoof belied the effort required of her to keep from laughing. "You didn't hear me knocking?"
It dawned on Sunburst at that moment just how long he had been working. "We were supposed to meet for lunch today. And I completely forgot. Sorry. Sorry. I'm so sorry."
"Mhm. Well, I know how you can get caught up in your research." Starlight walked over to the table, her eyes curiously wandering across the spellbooks and scattered pages that covered its surface. Most of the books were open to simple spells that most foals learn to cast with little more than a bit of verbal coaching, though there were a few others that looked unfamiliar to Starlight—since when did spells do arithmetic? What drew her attention, however, was the artifact around which everything else had piled up. It was a small black cube about the size of her hoof with smooth, glassy faces. "Maybe you can explain what was so fascinating this time?" She asked as she sat down in Sunburst's chair.
Sunburst pulled a spare chair over and sat next to her. "It's an AC—Arcane Computer—or at least it's supposed to be, but I'm having trouble getting it to work right. I was reading through the latest publications from Canterlot Academy, and I saw one paper by Lovely Lace where she described a way of combining several spells together that would allow an enchanted object to cast them on its own according to a program structured into the enchantment framework."
Most of that flew past Starlight, but the mention of an enchanted object casting a spell on its own made her sit up straight. "It's not going to cast anything right now, is it?" Starlight eyed the cube suspiciously.
"Oh no. It just finished running before you came in, so it won't do anything until I cast the activation spell on it again. Watch." There was a brief spark of golden light around his horn, mirrored by the cube with its own crimson hue, followed by a shimmering green projection of a three-by-three grid that floated in the air above it.
Starlight craned her head from side to side, unsure what to make of it. "What's it doing?"
"Right now it's just maintaining an illusion spell while it waits for input. There's a telesthesia spell in there to detect interaction. Try it: touch one of the cells."
Starlight reached out her hoof and waved it through the middle-right cell of the grid. An "O" filled in the cell, and a moment later an "X" appeared in the center square. At Sunburst's encouragement to keep going, she waved her hoof through the middle-top square, placing another "O" there followed by an "X" in the top-left. Finally recognizing the game, Starlight put an "O" in the bottom-right. The AC responded with an "X" to block in the top-right, and the grid filled up quickly from there, finally displaying "Winner: None" in hazy green letters before the spell ended.
"It plays tic-tac-toe," Starlight deadpanned.
"Theoretically, it can play any game. But tic-tac-toe is a good game to start with because it's a solved game; either player can force a draw, and it's trivially easy for me to work out the best move in any given position, so I can know if it's playing correctly, as opposed to something more complicated like chess where I'd never be able to tell if any move is actually good even if it consistently beats me." Sunburst adjusted his glasses then added, "Which wouldn't be difficult because I'm not actually any good at chess."
Starlight gave her friend a sidelong glance. "I'll rephrase: You stood me up to play tic-tac-toe by yourself."
Sunburst grimaced. "When you say it like that, it sounds pretty bad. I'm sorry. B-but let me show you something." He cast the activation spell again, and the tic-tac-toe board hovered in front of them once more. This time he played the center square, and the cube played the top-left corner. Sunburst played middle-top, and the cube played bottom-left. Middle-bottom, and a line crossed through the column of O's in the center before displaying "Winner: O" and fading out.
Starlight was not impressed.
"It's not supposed to lose," Sunburst explained with a sheepish grin. "I was trying to figure out how to fix it."
Starlight scanned across the loose sheets of paper that littered the table. Her patience was wearing thin, and although she usually took care to be sensitive about her foalhood friend's limits when it came to correctly casting spells, the thought did occur to her to cast the spell herself to correct whatever mistake Sunburst had made when enchanting the cube and then leave just to spite him. Unfortunately, none of the spells she could see made any sense to her. In fact, most of them didn't even look like complete spells. The closest she could find was a tangled mess of glyphs and arcing lines that looked like several completely different spells had been stitched together. It certainly looked castable, but she had no idea how to predict its behavior. She sighed. If stacking this many spells on top of each other was what it took to make a brick play tic-tac-toe badly, then she had to admire the complexity of what Sunburst had accomplished. She might have to forgive him for losing track of time.
"Okay, so where's the problem?" she asked.
Sunburst rearranged the pages across the table, lining up a dozen of them to reveal that what Starlight had assumed was the entire spell was only one part of it, and the complete spell now made her head spin just trying to read it.
"This part here"—Sunburst tapped his hoof on one of the pages near the edge closest to them—"is supposed to play out the entire game in its memory so it can decide which moves are good and which are bad." He cleared his throat. "But when I cast it like that, the AC didn't make any moves. After a while I realized the cube was getting hot and I had to dispel it before it set the table on fire."
"So how did you get it play at all?" Starlight asked while she stared at that part of the spell, trying to make sense of why it was wound into some kind of fractal pattern. She felt like she was almost onto something when she realized it was a spell designed to cast itself, but before she could discern why, Sunburst covered it with a different page that looked more like it belonged in an algebra textbook instead of a spell.
"I think I got something wrong when I was casting the part that contains the recursive spell, and it got stuck in an infinite loop, so I thought that instead of having it play out every possible game to select its next move, I could come up with a way to score the board for any given position and then it would just select the move that resulted in the best score." Sunburst rubbed the back of his neck. "But the scoring methods I came up with are bad so the best score isn't always the best move, and that's why I can beat it."
Starlight moved that page off the table and went back to studying the recursive spell underneath. As her eyes traced along the lines leading off to other sub-spells, the whole thing stopped looking like a mess of spaghetti and she could actually recognize what some parts of it were doing: keeping track of the state, determining if the game was over, displaying the board, recognizing input, and—"Why is there a telekinesis spell in here?"
"What? Oh. When I was first trying it out, I thought it would be easier if it just drew on paper. That, um, turned out to cause problems..." Sunburst pointed toward the ceiling where a quill had been embedded almost completely into the wooden rafter.
Starlight's eyes widened. She wasn't sure she could do that herself with only a telekinesis spell even if she wanted to. "Maybe we should take that part out if we're not using it."
Her use of the word "we" made Sunburst's ears perk up. "That's probably a good idea," he said as he brought out a fresh sheet of paper and set to work rewriting that section of the spell.
Starlight returned her focus to the recursive spell. Something about it didn't seem right, and she started stepping through it in her head. So first it reads in the board, determines if the game is over, checks which player's turn it is, counts up all the open squares, then picks one to play and casts itself again, where it reads in—Ah-hah! "I think I know why it got stuck. Look here: when it casts itself, it's still looking at the same board, not the one with the new move in it."
Sunburst had just finished replacing the page that previously held the telekinesis spell and he leaned over to look at the part Starlight was pointing to. His eyes wandered back and forth across the spell, following its branching curves until he came right back to the start. "I can't believe I missed that!" He groaned and facehoofed.
"Shouldn't be too hard to fix," Starlight said as she took the quill in her magic and wrote in a simple glyph and a curve leading back to the move being considered. She sat back and ran through it again aloud. "Now after it picks a move to try, it casts itself again with the new board, picks a new move for the other player and so on until either there's a winner or no more moves. Then it starts working its way back up the chain, assigning each move a score based on if it terminates in a win, loss, or draw, and decides which move to play based on that."
"Exactly! At least that's how it should work..." Sunburst nervously tapped his hooves on the table. "Do—um—would you like to try casting it?"
"I can give it a try, but I've never cast anything like this before. Even time travel only needed a single page, but this is like casting twenty spells at once. I'm not even sure where to start." Starlight glanced over at her friend and saw him chewing his lip while staring at the cube in the center of the table. "You've been re-casting this thing all day. You go ahead. After you have it working, you can show me how."
Sunburst took a deep breath and turned his gaze back to the spell, reading through it one last time before his horn lit up with the yellow glow of his magic. The spell was complicated, certainly, but in a way that seemed to make it easier for him to cast. Spells which had given him trouble in school had seemed to be doing something that left him feeling as if he were always out of step in a dance that he couldn't hear the rhythm to. But the AC framework didn't have that. There was a structure to it that let him take his time to get it right without fizzling out halfway through, as if he were describing the spell instead of casting it.
The cube shimmered with crimson light briefly as Sunburst completed the enchantment and a moment later it projected the board into the air above it. This was the moment of truth: Sunburst waved his hoof through the center square and it filled in with an "O". The AC played a move—already better than his first attempt with the recursive sub-spell—in the top-left corner. Sunburst continued with the middle-top, as his scoring hack had given the AC a habit of preferring corners even it meant losing, but this time it correctly blocked his column with an "X" in the middle-bottom. He quickly played out the rest of the game, barely containing his giddy excitement as the game ended with "Winner: None."
Just to make sure it wasn't a fluke, he and Starlight took turns playing against the AC several more times. Each game ended in a draw; it was impossible to win against it.
"This is a bad idea," Sunburst said. "The AC is going to lose!"
"It's your idea," Starlight reminded him. "You said yourself we need to find somepony who's good at chess to properly test our new spell." She raised the AC in her turquoise aura, admiring the glint of sunlight off its smooth faces. "Where better than here?" she asked, waving her hoof at the row of stone tables before them in the park where a dozen ponies were already gathered, playing friendly games of chess against each other.
"Yes, eventually. But we haven't worked out all the kinks yet. I still haven't figured out how to solve its problem with the en passant rule, and—more importantly—what if the telekinesis glitch happens again? Somepony could get hurt!" Sunburst winced as he absently rubbed his shoulder.
"We fixed that! Don't worry. And we'll just have to hope that the on pass-whatever thing doesn't come up." Starlight dismissed his concerns with a wave of her hoof.
Sunburst glanced quickly across the row of tables. "It looks like there aren't any open spots. Oh well. We'll just have to come back later." He grabbed for the AC with his magic while turning to leave.
"Somepony just got up! Come on," Starlight said, trotting out to claim the vacant seat. She kept a strong grip on the AC in her aura and Sunburst found himself dragged along with it by his horn.
An older earth pony stallion with a red coat and brown mane with a few streaks of gray running through it was sitting on the other side of the open table. He eyed the two unicorns curiously as they approached. "Hello," he said in a thick Stalliongrad accent. "You are new here."
Starlight sat down across from him and placed the AC on the table. "Yes, it's our first time. My name is Starlight Glimmer, and this is Sunburst," she said, motioning to her friend who was anxiously hovering over her shoulder.
"Kanterov," the stallion introduced himself. "What is this?" he pointed to the AC.
Starlight put on a broad smile. "This is an Arcane Computer, and it's why we're here. You see, we've been experimenting with creating a spell that can play chess by itself, and we were hoping to find somepony to help us test it. How good are you at chess?"
Kanterov snorted, holding back a chuckle. "I am rated twenty-eight-hundred." He leaned forward as he started setting up the board, giving himself the black pieces.
Starlight glanced at Sunburst who just shrugged. "Is that good?" she asked cautiously.
"Is good." Kanterov smirked. "Show me what magic box do."
Starlight cast the activation spell and the cube lit up with its own crimson aura. An illusion spell displayed a small blue circle in the air above it, and a bright band began spinning around it slowly. After a few seconds the king's pawn lifted up in its crimson grasp and moved two squares forward.
Kanterov quickly responded by pushing a pawn out two squares in front of his queen's bishop and recorded the opening moves on a fresh sheet in his notepad: 1. e4 c5. "This my favored opening," he said while leaning back for a stretch. "This does not bode well for—" He sat up straight as he saw the AC play the uncommon move c3, advancing a pawn to where it would soon disrupt his preferred sequence of moves. Furrowing his brow, he responded by pushing his queen's pawn to d5, where it was immediately captured by the AC's king's pawn, and Kanterov re-captured the pawn with his queen. He had barely written down the sequence of moves before the AC moved its queen's pawn to d4, asserting its control over the center of the board.
"I hope you are not here to make fool of me," Kanterov said as he developed his king's knight. "Some rival of mine perhaps playing remotely through your toy?" He stared intently at the spinning circle above the cube while the AC mirrored his move with its own knight.
Sunburst blurted, "I swear we have no idea who you are." He shrunk back behind Starlight as the older stallion's stare snapped to him.
"What Sunburst means is we're just here to test our spell. It beats both of us easily, but as I'm sure you can tell, we barely know how to play, so we have no idea how good it is," Starlight explained calmly.
Apparently placated, Kanterov returned his attention to the board and continued playing silently, taking a minute or two for each move as he methodically developed each of his pieces, undaunted by the AC's rapid responses. Certainly, none of his rivals would spend so little time calculating their moves against him. Near the end of the opening the AC had already castled while he had yet to do so, but he considered it acceptable since, after a trade of pawns had left the AC with an isolated queen's pawn—a weakness he knew he could exploit—he was able to clear a path to safety for his king and finish his development with an instigating move of his dark square bishop to b4, aiming through the gap in the AC's pawns.
The AC challenged the bishop by moving a pawn to a3, forcing Kanterov to retreat to a5. It didn't respond immediately after this, and Sunburst began pacing, worried that it had gotten stuck on something. He kept glancing between the board and the circle above the cube; as long as it was still spinning, the spell hadn't fizzled, and all he could do was wait. It may have taken only a minute, but it was a shockingly long minute before it finally moved its knight to c3, attacking Kanterov's queen which had sat motionless in the center of the board since the third move.
Kanterov didn't want to capture the knight, as it would cost him his bishop which he knew would be more valuable than a knight in the endgame, so his queen had to move. After taking several minutes to think, he moved it back one square to d6 and patiently considered his options for counter-play while waiting for the AC's next move.
The AC was taking longer to make its decisions now, and neither Starlight nor Sunburst knew if this was a good sign or not, though Sunburst was starting to worry if it was going to overheat again. At least the stone table wouldn't be at risk of catching fire.
When the AC decided to chase the queen by moving its knight to b5, Kanterov looked up from the board, raising an eyebrow as he locked eyes with Starlight. Was it a good move? Was he surprised by it? Afraid to ask, she just smiled.
Kanterov made an impassive grunt and returned his attention to the board. He spent a long time calculating before ultimately retreating his queen to e7. After another couple minutes of waiting, the AC moved its other knight forward to e5, taking advantage of the fact that Kanterov had still not castled by introducing a threat against his queen-side knight on c6 and simultaneously revealing an attack against his light-square bishop on h5. The knight was not an immediate problem, but the AC had both its queen and bishop lined up against his bishop which had only the king's knight to defend it. With nowhere to retreat, he was forced to exchange bishops with the AC and with the pressure momentarily relieved, he finally castled.
The AC continued its attack by moving its queen's rook to c1, adding pressure to Kanterov's knight. He was now faced with a problem: if he traded with the AC's knight, it would bring that isolated pawn into an attack against his remaining knight. Beyond that, the AC had coordinated its pieces so well that from that position it would then be able to win a pawn from him and even bring its rook forward to threaten his queen. Retreat and defend seemed to be the only moves the AC would allow him to make.
Kanterov mirrored the rook move to protect his knight and stood up to stretch his legs and take his eyes off the board for a few minutes of rest while he waited for the next move. It was at this point he realized a small crowd had gathered around his game against the unicorns' magic box. They were standing a polite distance away and keeping quiet, but he could hear them whispering speculations about the strength of his position.
"Mister Kanterov?" Starlight's voice caught his attention. "It's your move when you're ready."
He sat back at the table and took a deep breath as he took a fresh look at the board. The AC had played its dark-square bishop to g5, pinning his king-side knight to his queen. Doing this had removed a defending piece from the isolated pawn on d4, leaving only the knight on b5 to protect it. He was, however, still facing down a very aggressive position where the AC had managed to constrain his pieces to his side of the board. He had precious little room to maneuver. Kanterov needed to begin a counterattack, or he would quickly find himself with even less to work with, so he moved his bishop to b6, focusing on that weakened pawn in the center. All he had to do was win that pawn, and the game would shift in his favor.
The AC spent several minutes calculating from this position. During this time Sunburst found himself drawn aside by some of the onlookers. While the crowd was hardly interested in the AC's enchantment framework, there were many questions about how it decided which moves to play and how many moves deep it was able to calculate. Sunburst did his best to answer them without getting too technical and being careful not to mention any of the flaws he knew it still had.
That didn't stop some of the ponies in the crowd from asking questions that seemed to be aimed at finding possible flaws. As one of them, intrigued by the fact that the AC's calculations assumed its opponent would always chose the best moves it could find for them, asked, "What if Kanterov played a bad move? Wouldn't that make it think it made a mistake and that the move is actually good?"
Sunburst blinked. He knew very well from personal experience that the AC had no problem defeating bad moves. "No... It doesn't think. A bad move is a bad move and it will see it that way. You can't fake it out like that."
There was some murmuring within the crowd at that. Some of them took issue with the idea of something that couldn't think being able to play chess at all, let alone well enough to challenge Kanterov.
Then came the question Sunburst had hoped wouldn't be asked: "Is it going to win?"
Sunburst had no idea if the AC would win and, as he had come to realize how strong of a player Kanterov was, worried what it would mean if the AC could defeat him. He was stammering his way through an evasive answer when the sound of a piece being set down on the board drew everyone's attention back to the game. Sunburst breathed a sigh of relief and excused himself from the crowd, returning to Starlight's side to see what had happened.
The AC had moved its bishop to f6, capturing the knight that had been pinned there in front of Kanterov's queen. This had put him in a precarious position, but the possibility of this move had not escaped him. Kanterov knew that he couldn't recapture the bishop with his queen because the AC's king-side knight would then move to d7 to attack both his queen and the rook guarding his king; he would lose the rook. Unfortunately, this meant his only viable option was to capture with the pawn directly in front of his king, ripping open his castle.
The AC responded immediately by moving its knight to c4, getting it out from under the attack by that pawn.
Determined to press his attack against the isolated pawn in the center, Kanterov moved his king-side rook to d8. The AC followed by trading its knight for the bishop on b6, forcing him to double-up his pawns in the b-file to capture the knight.
After a minute, the AC decided to defend its weak center pawn with its king-side rook, and Kanterov responded by pushing a pawn to f5, opening up a diagonal for his queen and clamping down on the space available for the AC's queen. It moved its queen forward one square to where it aided in the defense of that pawn, and Kanterov followed suit by moving his queen onto the open diagonal, adding to his attack on the center. That single pawn was now sitting in the crosshairs of a knight, rook, and queen from both players.
And then the AC moved that pawn.
A hushed chorus of gasps and murmurs echoed through the crowd at this move, and even Kanterov appeared surprised by it. Starlight and Sunburst exchanged glances with each other, neither understanding why a simple pawn move had received such a reaction.
Kanterov knew exactly why. Up until this point, the AC had played very materialistically; it never allowed him to capture a piece without itself capturing something of equal value—pawn for pawn, bishop for bishop, bishop for knight, knight for bishop. But with this move, it had effectively sacrificed the pawn. It was a move he would have played in its position.
He won the pawn with a trade of rooks, but was now presented with a new threat from the AC: both its queen and rook were on open files that would allow it to directly attack his king. In preparation to defend himself, he moved his king into the corner on h8 and after the AC captured the undefended pawn on b6, he repositioned his rook onto g8, taking advantage of the opening in his castle to threaten the pawn directly in front of the AC's king.
The AC appeared to have returned to its materialistic behavior, moving its queen to c5 to attack another undefended pawn, seemingly unconcerned about its own king.
Kanterov advanced his pawn to d4 where his knight and queen guarded it, and the AC brought its knight to d6. Against any other opponent, Kanterov would have considered this a mobilization toward his king, but now that he had seen the greedy nature of these unicorns' bauble, he assumed this was only because it was threatening two pawns at once with this move and he continued his attack by advancing his pawn to f4, closing in on the AC's castle.
The AC confirmed his suspicions when it moved its knight again to capture his b7 pawn which was of no practical value at this point in the game. Kanterov laughed at this. "I'm not so sure you're on right track now," he admonished as he mobilized his knight to e5, seizing the opportunity to build more pressure against the AC's king.
Sunburst felt a rush of lightheadedness while Starlight bit her lip to keep from making an outburst. Neither of them could tell on their own how bad the position really was, but Kanterov's confidence told them they were about to lose.
But the AC was immune to such psychology. It knew a good move from a bad one, and it knew when victory was certain. Its queen moved to d5, guarding against a possibly dangerous check from Kanterov's knight. That didn't, however, stop him from pushing his pawn to f3 instead; a move that threatened to break the line of pawns guarding its king. Advancing a pawn to g3 maintained its defensive structure, if only barely.
Kanterov's knight pivoted to d3, threatening the AC's rook which then abandoned its king in a move to c7, coordinating with is queen in a counterattack. There was no threat of check that came with such an attack, however, and he ignored it to move his rook onto e8 with a clear path to the AC's unguarded back rank. Unless it did something to stop him, he would have checkmate in three moves.
The AC played its knight to d6, attacking the rook, but doing nothing to stop him from charging his rook across the board to attack the AC's king.
"Check," Kanterov announced calmly.
The AC had only one legal move and retreated its king into the corner. And after its f2 pawn was captured by Kanterov's knight, it was one move away from losing. But Kanterov had overlooked a fatal weakness in his position, and when the AC's knight attacked his king, his eyes widened as he realized his mistake.
A sudden quiet fell around the table. Starlight looked across at Kanterov and cautiously said, "Check."
He quickly moved his king.
The AC moved its knight, revealing another attack on the king from its looming rook.
"Ch-check," Sunburst said, feeling his heart race.
Kanterov moved his king again, but the AC pressed the attack with its rook.
"Check!" Starlight could barely restrain herself.
It was there that Kanterov, with a heavy sigh, toppled his king and stood up to reach across the table and shake hooves with Starlight and Sunburst.
"Can you be more specific?" Sunburst asked. "Make it better how?"
Starlight paced back and forth across the room while carrying the AC. Her eyes were fixed on the black, mirror-like surface of its faces as it slowly rotated in her magical grasp. As exciting as it had been to win a game of chess with it—and it had been especially gratifying once she and Sunburst learned that their unwitting opponent had been the reigning champion professionally for over a decade—she wanted to see just how far they could go with it. "Better! Just better, you know? Faster. Smarter. I don't know. There has to be more we can do with this thing."
Sunburst levitated his glasses off his face and cleaned them with the corner of his cape while he tried to think of something Starlight would consider "better." Since their victory in the park earlier in the week, he had wanted to fix the problems he still knew it experienced under certain conditions, but Starlight wouldn't hear a word of it; they had conquered chess as far as she was concerned. "I think there are other games considered more difficult than chess," he sad as he set his glasses back on his face. "We could—"
"No. No more games. This thing can cast any spell we can, and it can make decisions we don't even understand the reasons for. There has to be something more useful we can do with it." Starlight stared at her reflection in the AC's surface, and her eyes widened as a thought came to her. "What if we gave it a body?"
"Games have specific boundaries and well-defined rules. They're easy to work with, and if it's doing things we don't understand under those conditions, is it even a good idea to—Wait. A body?" Sunburst blinked. "What does that even mean?"
Starlight trotted over and set the cube down on the table in front of him. "What if it looked like a pony? What if it could talk to us?" She gasped as the idea began to flourish in her mind even as she raced to explain it. "What if we could talk to it? Give it instructions verbally, and then it would figure out the best way to do what we tell it."
Sunburst adjusted his glasses with a hoof while he struggled to keep up. "Well, I suppose the illusion spell can be expanded to project the image of a pony, and I think I noticed Trixie using a spell to throw her voice the last time you brought me to one of her shows; that could be adapted so the AC could make sounds... The telesthesia spell can already pick up on sound, so in theory we can program it to respond to voice commands..."
"Let's do it!" Starlight bounced on her hooves with a wide grin on her face.
"O-okay... But what commands do we want to program it for?" Sunburst moved the AC aside and pulled out a fresh sheet of paper and a quill to start diagramming the new enchantment matrix. He was already certain he wouldn't be able to reuse anything from the chess spell, but that was somewhat of a relief to him as that had quickly become a patchwork mess of quick-fixes for fundamental flaws in the spell's architecture. Starting from scratch would give him the opportunity to correct those mistakes.
"All of them," Starlight said. "We should be able to tell it to do anything."
Sunburst sat motionless for a moment before he put his quill down. "I have no idea how to do that. There's no way we can encode every possible thing we could say into it. Or even if we could do that, how would we even model the world so that it could calculate the effect of its actions?"
Undeterred, Starlight pulled up a seat next to him and started writing the illusion spells she already knew they would need. "We'll figure something out. Or maybe somepony else already has. Have you checked the latest publications on arcane computing? Maybe there's something we can learn from."
Sunburst tapped his chin. The field of arcane computing was still very new, but the academic circles were finding new uses for it at an accelerating pace. It was quickly reaching a point where Sunburst wouldn't be able to keep up with it, especially as other disciplines were getting involved. He had even seen a reference to a biology article... His eyes widened as he jumped out of his seat and began digging through his collection of cheaply-bound research journals. "Ah-hah! Starlight, what about this? This might solve all our problems," he said as he carried the journal back to the table and opened it to an article titled Algorithmic Learning in Arcane Computation by Graceful Hop.
Starlight glanced at the article. There was a diagram of circles arranged in "layers" with arrows pointing between them but it didn't look like a spell. She flipped through a few pages and came to a stop on a page filled with... Are these equations? When did they start doing math without any numbers? She furrowed her brow and eyed Sunburst suspiciously. "What am I looking at?"
"This article's about the application of a mathematical model originally developed to describe how neurons in a brain work. If we implement this, we won't have to program the AC to understand our instructions or even how to model the world around it; we just have to give it a bunch of information and it will learn on its own." He grinned widely as he sat back down and started over on a new architecture for the enchantment matrix. His excitement quickly faded as a new problem came to mind. "It's going to need a lot of information to work from, though. A whole library's worth at least, and I'm not sure how we'd give it that."
"I think Twilight mentioned once that she'd experimented with a modified dream spell to learn from a book without actually reading it. Could we use that?" Starlight asked.
It didn't surprise Sunburst to hear that Twilight would have a spell like that. "That sounds almost perfect. Would you ask her for a copy the next time you see her?" he asked as he started incorporating an interface for such a spell into the design. After a few minutes of continuing to work, Sunburst realized he hadn't heard a response and he looked up to see that he was alone. "Starlight?"
There was a brief flash of light accompanied by a pop as Starlight teleported back into Sunburst's study, triumphantly waving the scroll she carried with her. "Got it! And Twilight said we're welcome to use her library. That should be enough for the AC to learn from, right?"
Twilight's library was indeed enough to learn from. At least if it weren't, then Sunburst didn't think there were enough books in existence to train the AC.
While it had taken him and Starlight the better part of a week to write their new spell and enchant the AC with it, that time had passed in a blur. Running the learning algorithm was a tediously slow process by comparison, however. Once they had set up in the library and activated the AC, the only thing for them to do was sit and wait.
Every few seconds, the AC would briefly light up with its crimson aura as it cast Twilight's book reading spell on another book somewhere in the library. Even with how quickly it read each book, they had been in the library for hours already with no idea how much longer it would take.
"Do you think it understands what it's reading?" Starlight asked dryly, her eyes closed and head propped up on her hoof.
Sunburst was slumped over the table with his head down on one of the pillows Spike had brought them a while ago. "No. It's just doing math to form associations between symbols so it can predict"—he covered his mouth as he yawned—"predict which symbols are likely to occur together and make decisions based on that."
With one half-open eye, Starlight glanced across the table at her friend. "How is that different from understanding?"
Sunburst sat up slowly and took a moment to clean his glasses while he considered an explanation. "Imagine you're locked in a room, and every so often somepony slips a note under the door written in Ancient Ponish," he said at length. "You don't understand Ancient Ponish but you have a set of instructions that tell you what to write down as a response for any given input. It could be the key to unlocking the deepest mysteries of the universe or a recipe for beetroot stew for all you know. Anypony outside the room would be convinced that they're talking to somepony who understands Ancient Ponish, but they're not."
Starlight shifted her head to her other hoof, both eyes open now. Something about that argument didn't sit right with her. "What about me and the room as a whole? Does that count as understanding Ancient Ponish?"
Before Sunburst could answer, the AC made a short chime. It had finished reading every book in Twilight's library. The two unicorns stared at it silently while the cube sat inert on the table. A week's worth of work, the whole day spent waiting, and now they would finally know if it worked. All they had to do was cast the activation spell.
"Do you want to... or should I?" Sunburst asked.
"I can do it—I mean, that is unless you want to?" Starlight asked as she moved around the table to stand next to the AC, and Sunburst gave her a nod. She shuffled on her hooves for a moment. "I guess I should mention that I made a small change to the spell before we enchanted it."
"What kind of change?" Sunburst felt a prickle of anxiety crawl its way up the back of his neck.
"Nothing really. I just thought it should have a name so it knows when we're talking to it, so I wrote one into its voice command spell: UniVAC," answered Starlight.
Sunburst ran through that sub-spell in his head and nodded slowly as he concluded that it shouldn't interfere with the rest of the matrix. "That's a good idea. But kind of a strange name, isn't it?"
Starlight shrugged. "Well, I figured it should be unique so it won't get confused by random conversation. It's short for Unicorn-Visualized Arcane Computer, since it'll have a body now." She took a deep breath as her horn lit up with the activation spell. "Here it goes..."
The cube shimmered with its crimson aura for a few seconds before the form of a unicorn with a beige coat, a matte silver mane, and black eyes with red pupils that stared blankly straight ahead appeared in the air above the table where it floated, motionless.
"Okay, we'll have to get it to project the image onto the ground, but so far so good," Starlight said, sharing a small laugh with Sunburst. "UniVAC," she said, addressing it. As she did, the eyes began to pulse with a dim red light, and the illusion turned its head to face her. Both she and Sunburst made a sound of disgust as they watched it move. There was something unnatural about the motion; it was much too fast, as if it had skipped directly to that position.
Starlight cleared her throat and cast a sly glance at Sunburst as she thought of what to ask it. "UniVAC, do you understand me?"
The illusion's head snapped back to its neutral position and without opening its mouth, began speaking in a monotone voice, "Chapter one: the animal cell. All the tissues of the body originate from a microscopic structure, the fertilized ovum, which consists of..."
"I think that's a 'no.' What's it doing?" Sunburst shouted over UniVAC.
"...a soft, jelly-like material enclosed in a membrane and containing a vesicle, or a small spherical body inside..." it continued.
"It sounds like it's reciting a biology textbook. UniVAC, stop!" Starlight waved her hoof at it, but it didn't respond. "Why is it doing that?"
"I don't know! Just turn it off!"
"...this may be regarded as a complete cell."
Starlight cast the deactivation spell and the image vanished. She and Sunburst shared a glance at each other and sighed. This was going to be more difficult than they had expected.
