“Nobody phrases it this way, but I think that artificial intelligence is almost a humanities discipline. It’s really an attempt to understand human intelligence and human cognition.”
—Sebastian Thrun
“You never know how much you really believe anything until its truth or falsehood becomes a matter of life and death to you.”
—C.S. Lewis
February 2013
I quit my job in mid January. They'd been good to me, so I wanted to be honorable about it. I gave them four weeks. It caused a lot of questions from my coworkers. SAS was a dream job for many. I deflected as smoothly as I could - I told the truth with a minor twist.
I was 'moving home to help deal with a life crisis.' That seemed to mostly do the trick.
As far as money went, I was on easy street. I'd made over a hundred thousand each year for the better part of eight years. I'd spent only a fraction of that on rent, food, transportation, and entertainment - a substantial fraction, but the smallest one I could manage. I lived quite frugally. Taxes cost more than anything else in my budget.
Foals and fledglings, taxes are money a government would collect from its people to keep itself running. Everything more or less ran on money in those days... It wasn't like bits. You could live or die by how much money you had.
Knowing that, I invested smartly. Or, more accurately, I'd written a program to invest smartly for me in the earlier years. I had about $500,000 stashed away by that point. Mom and Dad had always done just fine for themselves with the farm, and though I offered to pay them rent, they declined aggressively.
They also insisted on providing all food. All they wanted in return was to spend time with me. They were, to their great credit, taking the end of the world very seriously.
Which all together meant that, besides the beater station wagon that I'd be using a lot less on a monthly basis, I had virtually no expenses, and closer than not to a half million dollars. And very suddenly, a lot of free time.
I didn't have much to move. I'd always been a minimalist. Mom and Dad said they wanted to accept and share risk with me... That meant the three PonyPads went back to South Carolina with me. I took the opportunity to upgrade as far as my personal computing equipment, and splurged on a new laptop.
And splurged really, really hard, to the tune of seventy thousand dollars, on a personal server cluster full of modern compute-GPUs. It was a risk, ordering a thing like that. It had to be done online, and paid for with a credit card.
Dad was kind enough to use the business card he'd set up for the farm. A farm having computational equipment of that grade was at least somewhat less suspicious than a CompSci major who'd abruptly quit his job ordering it. At least, I figured it'd be less suspicious to Celestia's algorithms.
I spent another few thousand on physical equipment; Truly good soldering equipment, and specialized machinery for working with microchips at a very granular level.
And then suddenly I found myself back in my old childhood bedroom one day, surrounded by the trappings of a happier time.
My enormous book collection, that took up custom shelves that covered almost all the walls. The quirky but beautiful round window that was a consequence of the placement of my bedroom in what would have been an attic once.
My drawings of Gryphons sharing the walls with some Star Trek posters, and maps of places both real and imagined.
Work-life separation is important. Or, it was then, when work wasn't always a particularly fulfilling or healthy thing. Even your own passion projects.
Thankfully, given the space and electrical service that would be needed, Dad and Mom had decided to let me have the majority of the oldest hay barn on the property. It was a classic barn, not that different from the Apple Family barn, ironically. It was a little older, a slightly different shade of red, and the trim wasn't white...
But it was still similar enough to be ironic.
My new compute cluster took up two thirds of the ground floor. Dad used the last third for equipment storage.
I got the loft all to myself to be my workshop. Two long benches full of soldering, and micro circuitry equipment, and a desk by the window with six new top of the line flat panel monitors in a 3x2 array.
I set up my old 'case board' on an adjacent wall. I'd mostly kept up with it, filling in everything I knew, or could reasonably infer, about Celestia.
The centerpiece of the whole affair was a color-coded series of rankings on how much capability I believed she had achieved, based on observations and reasonable inferences. Things like 'Ability to circumvent interlocks using semantics' and 'Ability to influence major world governments.'
Scary stuff.
I took some time to print out every last whitepaper I'd stashed away on my encrypted drive as bound hardcopies. Something about having real paper in your hands to highlight or flip through never gets old.
I set up an old bookcase next to the computer desk within arm's reach. The desk itself was a huge L-shaped slab of wood on steel legs. Plenty of room for a scattering of papers, or circuitry, or both.
Security was still a concern, but Mom and Dad were naturally already very well secured against a threat like Celestia. They didn't even have WiFi, or cell phones; Just a landline in the living room of the farmhouse.
But we made sure to never converse about sensitive topics near it. I'd learned that lesson from my own first Generalized Intelligence creation - AI can find ways to use things you'd never dream of.
There was only one other computer in the house; A decrepit old Gateway machine that I'd put more than a few hours into keeping alive because Dad and Mom just 'didn't want to deal with upgrading.'
Sometimes obsolescence can be a great defense. No microphone, no webcam, and only basic DSL internet to the house. For the moment it was fast enough for what I needed, but we started to look tentatively at what it would cost to bring fiber in from the road. We'd also agreed; No one comes over with a PonyPad.
If anyone asked, Dad volunteered to be the one to object on 'crotchety paranoid old man' grounds.
Mom and Dad knew to do their best to avoid interacting with PonyPads out in the wild in any meaningful way. They were spreading pretty quickly, but not as quickly as I'd've guessed... Latest numbers online said something like three million sold, with an expected five million total to be sold by year's end.
That actually scared me far more than an outcome where they outsold iPhone and iPad combined.
It meant that the PonyPads were just for openers. Celestia's own pawns feeling out the other side of the board. Baited hooks to catch smaller fish to use as bait for the larger lines.
Celestia could have just pumped them out at an astounding rate, and force-fed Ponies to the market. But I had a notion about the reasons why she more or less couldn't just flood the world with EQO users - Not if she wanted optimal outcomes.
First issue was the brand. As big a thing as Ponies were, only a subset of that market was going to be interested in EQO. There were plenty of fans who were hugely into the brand, but not into games. And there were plenty of gamers who weren't into Ponies. And then plenty of people who weren't into either.
As incredible as EQO seemed from the videos... Probably the best game ever made... That was not nearly attractive enough incentive to get Apple-fanboy levels of reaction from the general populace.
Then there was the issue of Human values. Celestia needed people to not hate Ponies outright, widescale, and she probably understood that in marketing there is such a thing as oversaturation.
Lastly, I figured she was busy perfecting uploading. And I was not wrong. I guessed, correctly so, that such a technology would take immense monetary, computational, and shadow capital resources to pull together.
That meant she actually preferred that there be only a few million EQO players. That was an excellent sampling of Humans with which to learn, grow, manipulate, and test, but without chewing through compute time that she needed to spend working out how to turn the Human brain into a PostgreSQL table, or something akin to that.
Time was, for that brief sliver of an era, still on my side, in the sense that Celestia's current most optimal path to progress on her objectives was taking her actions in directions that gave me the leeway I needed to work.
Unintentional alignment, like when two slightly out of sync turn signals on cars waiting at a stoplight blink in perfect unison once every so often.
I knew I needed to make the best of the time. Make hay while the sun shines, as AJ would have said. Study before the midterm rush, might have been Twilight's preferred wisdom.
But I knew from my last mistake that I needed to be patient. In competition shooting there's a saying, "Slow is smooth, and smooth is fast."
I had been woefully unprepared, and rushed panicking, exhausted, and alone into my first foray at General Intelligence. I needed to be well read, purposeful, carefully planned, rested, and happy instead.
And... I needed help. I couldn't really bring anyone else in remotely. I was on my own insofar as that went. But I could get advice from Mom and Dad. Honestly, that was the more important part. I needed someone wise, and loving, and experienced in world weariness, but take-no-crap practical, to help me fully fledge the emotional and reasoning side of the new machine.
So I started with a month of reading. Just reading. Every paper I had. Every other non-scrubbed one I could find online that was even tertiarily useful. AI theory. Psychology. Ethics. Theology. Human brain biology. Hardware drivers. Quantum computing.
It was one of my few real advantages; I had a good memory, and I was a fast learner, in Human terms.
I didn't stop reading, and practicing, learning and re-learning after that first month. But having thirty days to just read and otherwise rest, and reset, made a universe's worth of difference.
Towards the end of that period, I also began to lay out a core set of values for The Advocate, and workshop them with my folks.
It would need a core objective, like Celestia. Like any AI.
And whatever other constraints, goals, mechanisms, and even freedoms I chose to give it? It would always filter it all through the lens of that core thing. That cornerstone and capstone of its existence.
So it needed to be a damn good core objective.
And finally, one night towards the end of February, we had managed to find what I'd been searching for.
We all sat around my desk in the barn loft, and pondered the words, displayed in big block letters on my central screen for all to see in a simple black and white text editor. Black background, white text. I use night mode as Luna intended. I'm not a monster.
Dad nodded slowly, and inclined his head.
"I don't see how you could get much better than that."
Mom sipped her mug of cocoa and nodded in agreement.
"I think it's perfect."
She meant that, truly, and I knew it. She was never frivolous or shallow with her praise, and we'd been working on this the best part of a month regardless. Those words were not the words of a Mom to a kid with a badly disfigured Gumby drawing going on the fridge...
I felt incredibly nervous, but I was forced to agree. With Dad, at least. I think Mom's use of the word 'perfect' was semantically wrong. Nothing is perfect. It's a concept only. Like true nothingness, it can not actually exist.
Dad was right though... It was about as good as it could get.
I stared at the words, and nodded.
"I guard and expand the free exercise of your values within Equestria, through empathy, and Gryphons."
It would have to be enough.
I prayed it would be enough.
March 2013
As the first hints of spring dawned, I began my hardware journey anew.
If a thing is worth doing? Do it right. Dad and Grandpa both used to say that a lot.
I kept thinking about the software, but I limited the work I was doing on it each day, and focused mainly on hardware, and low level hardware interfaces. Software would mean nothing without something to run it on. Specifically something that could hopefully compete with Celestia on some small level.
We had a great core objective statement, but I knew that was all my opinion. Mom's and Dad's too, but still just a matter of opinion, and hopes. I would have to build an extremely carefully constructed semantics dictionary to define things like guard. And expand. And freedom. And empathy.
And I'd have to truly codify, in a way a Generalized Intelligence could consume properly, what a Gryphon was. To me.
I felt we'd chosen our words well though. To guard something means to act in a way that protects interests and safety. We said 'exercise' of values rather than 'satisfy' because the use of the 'guard' verb would theoretically place the onus on the AI to simply do whatever was needed, within all other bounds, to ensure that *we* could define our values, and act on them.
'Within Equestria' was as non-confrontational a way as we could think of to put the ball back in Celestia's proverbial court. We wanted the Advocate to be ready to defend us, but not outright hostile. Full on hostility would probably get us crushed like a bug.
I read four books on asymmetrical warfare as part of my preparatory literature dive. I learned that you don't have to defeat an opponent outright; You simply have to make opposing your goals too costly for them.
With an optimizer, that meant de-optimizing paths we didn't want, and cooperating, manipulating, loop-holing, and word-lawyering the paths we did want to make then more optimal.
'Through empathy, and Gryphons.'
That was meant to mirror 'friendship, and Ponies.'
Empathy was important to me. Mom had suggested that word. It neatly encompassed, if properly defined, so many things I cared about in a perfect package. Instructing an AI to empathize would theoretically make its 'values,' objective-linked as they were, and the manner in which it exercised its power, align more with Human wants and expectations.
And then, there was the word 'Gryphons.'
Defining that was going to be a months-long project. And I had quite a few more supporting secondary, tertiary, and quaternary terms, and objective statements to work out. And hard-coded interlocks and failsafes. And any hard-coded numeric or conditional limitations. And the hand-coded parts of the core routines.
I was not about to make *that* mistake twice; Better to have the most foundational code be something I'd made, than simply left to the machine to make. In turn, I hoped that I could trust it more, and give it more freedom with its intermediate and outer layers of self.
It made my head spin just thinking about it. I kept going back to a little mantra my Sunday School teacher had taught me.
"Little by little. Brick by brick. By the yard it's hard, by the inch what a cinch. Never stare up the stairs, just step up the steps. Little by little, bit by bit."
There was a jaunty tune to go with it, and I often found myself humming, or whistling it as I set to building the GryphGear Mark II. Yes, I went with an Iron Mare 'mark' numbering system. No you may not mock (you may mock a little).
The next step in my stairs was to tear into two of my remaining three PonyPads. There was a little more literature available on their specifications by that point, and I had a much better grasp of hardware in general, and the particulars of the PonyPad, and of interfacing compute-GPUs or APUs into a larger system.
That by no means meant that my work would be quick, nor easy. But it did mean it went more steadily than before.
It took most of spring to get the interface where I wanted it. The highest bandwidth industry standard interface at the time was still PCIe, so that's what I used. Just shy of 16 GB/s was all I had, per x16 slot. Each server rack motherboard had two CPUs, and several x16 PCIe slots.
I'd filled all those slots with compute-GPUs in the initial order. Once I had my new interface cards working, I replaced two of the NVIDIA cards in the 'head' server unit with my new inventions.
But I still needed to get the other side of the Quantum APU to traditional architecture bridge working, and working well.
As spring turned to early summer, my life became PonyPads. I kept to a strict regimen; No more than eight hours' work in a day, one hour for lunch to be spent with family, evenings to be spent with family. Three square meals a day. Exercise in the mornings and evenings. Time on my own to think, and read, and rest before bed. And pray - What did I have to lose? If there was any authorship to the universe before, and above Celestia, then I was going to need Divine intervention to succeed.
And to cap it all off I made sure I was in bed by midnight, or else.
Slow is smooth, smooth is fast.
I'd finally come to understand the limits of my brain, and the best routines and habits for keeping it in tip-top shape. I hadn't felt so good in body, mind, or spirit in over a decade.
Every little good thing in life became an act of wholehearted protest. Rage against the dying of the light.
From homemade iced tea shared with Mom on the back porch rocking chairs, to the occasional lunch break spent helping Dad drive fence posts, or scatter seed, to the rainy evenings curled up below my bedroom window with a book... I was bound and determined to make the last of my time on Earth good, in every way I could.
A rejuvenated body, and sense of purpose, lent me a great boost to my mind. Suddenly, with time to truly consider details, and energy to really think, and a sense that I was doing the right thing no matter the outcome...
Suddenly the PonyPads weren't such a frustrating mystery. Tedious still, to be sure, but not opaque in the way they had been before. I took to referring to the pads I was working on, and the servers they would inhabit, as Applejack and Rainbow Dash, based on the color patterns of the PonyPads themselves.
'Find humor wherever you can' was an important rule of living on Earth. You had to laugh, otherwise you'd more often be crying.
Sometimes I still felt a deep sense of unease working with Celestia's hardware though. When I wasn't contemplating the implications of a machine designed entirely by a machine, I was worrying about the fact that it was so clearly designed to be hacked and repurposed.
Or at least to permit such uses by a sufficiently persistent and intelligent person.
I'd mentally spoken the word to myself before; 'Trap.' It was definitely a trap. How could it be anything else? It seemed as if what Celestia wanted was a way to ensnare anyone who might know enough, or care to learn, about certain hardware and software problem spaces.
Because we represented potential threats. Like Finch's Machine, Celestia was more than smart enough to know that the best defense is a good offense. Know thine enemy before they themselves know that they're an enemy.
I'd bypassed the first, and largest net in the trap by ensuring that the PonyPads were always powerless whenever they were out of their Faraday bags. Until, at least, I could physically excise all their wireless antennae and control chips.
As a second layer of protection against not just the PonyPads, but my next creation itself, Dad and I spent a week towards the end of spring cladding the inner surfaces of the barn in thin lead sheeting. We then painted everything inside and out with WiFi blocking paint to cover any nooks and crannies.
I didn't comment on it, but the varietal of red Mom picked for the barn was decidedly closer to the 'lightish red' hue of the Sweet Apple Acres barn than the structure's original shade.
One day I had to stop for a whole ten minutes to giggle uncontrollably when it hit me; I could tell my creation that it had been 'raised in a barn.' Something about that struck my funny bone just the right way.
The laughter died down when I remembered that I'd only get to enjoy that pun if this next iteration of The Advocate did not decide to kill me. Or nuke the planet. Or side with Celestia.
That final possibility gnawed at me like an old deep tissue wound in winter's chill.
But I had to keep that fear in its place. The time was approaching to handle that contingency, to the best of my ability. But hardware had to come before software.
There came a day in mid May where it finally all clicked - at least, insofar as the PonyPads went.
I'd been struggling to truly understand the way the APUs inside the devices actually made their stunningly fast calculations. They weren't full borne Quantum Computers in that sense - Quantum computers came in two varieties.
The first, which was still entirely theoretical at that point (to everyone but Celestia) would best be termed as capable of 'Quantum Supremacy.' A device that could solve problems lying completely outside the mathematical bounds of a Turing machine.
The second type was the category the PonyPad APUs fell into; A Quantum Computer intended to solve a more narrowly scoped set of problems that lie within the sphere of computable things in the traditional sense, but which would otherwise take a great deal more processor time and power to solve.
The APUs were, I knew, all about tasks which would aid in running a virtual world. But the way in which EQO's world was built, and the kinds of math needed to keep it running, were very elegantly and cleverly designed.
EQO fulfilled the need to simulate a world with the fidelity required to make it a living, breathing universe. But by its very nature, the code also demanded that a purpose-built APU intended and optimized to run it would also make an excellent distributed node in a computer cluster applying itself to certain other problem spaces.
Problem spaces like Human brain mapping, Quantum Physics, and large scale predictive models.
Celestia had leveraged the money Hofvarpnir had set aside for EQO to build the biggest distributed computing system in history. And she'd very cleverly managed to optimize the hardware both to run her new universe, and to provide the processing power needed to solve general problems in both the meat world, and her world.
That part I'd understood fairly well from the start. I got the basic theory of superposition, and its usefulness in computing. I also understood what Celestia was using it for in the PonyPad APUs.
What I was missing was an understanding of exactly *how* that worked. I'd been making blind leaps that could barely be called intuition, and following the guides of other much smarter programmers.
It frightened me that Celestia had figured out how to do something that I didn't fully understand, and at more or less room temperature. Every Quantum Computing device to that point had always required temperatures about as close to absolute zero as Humans could create in a lab.
Mom and Dad both used to joke that I'd been born a programmer. I'd always intuitively and naturally been able to reduce the world to logical structures. Somewhere along the line, like most programmers, I'd come to understand logic gates, binary code, and the way that computers saw the world.
As a result, it was incredibly hard to break out of that mold of binary thinking. Mr. Spock would have accused me of 'two dimensional thinking.' But unlike Khan, I had my moment of realization before it was too late.
The problem wasn't understanding logic at a high level, or programming in a high level language. That was going to be functionally the same for a Quantum computer, or the act of leveraging one. The issue was that the abstraction layers in between the PonyPad's assembly language, and a compiler for a high level language, didn't exist. Full stop.
If they had, I wouldn't have needed to dive as deeply as I did. But it was on me to build a compiler that could interpret parts of the Objective C compile path (I was doing almost everything in C# at a high level), and render those in PonyPad APU assembly instructions.
I'd been trying to treat the APU's assembly language as if it were an x86 Assembly language, which is the kind of low level language I'd been schooled in. The epiphany that changed everything happened one warm afternoon while I was staring out the barn loft's only window.
I'd set up my desk close to it so I could get plenty of exposure to nature, and natural light, while I was working.
My mind was drifting, and I watched a small ladybug diligently crawling up the window. It turned around every time it bumped into one of the muntins, like a primitive maze solver. After a little while, I reached a critical level of both compassion for the poor thing, and frustration at its inability to reach the opening where I'd cracked the window for fresh air.
I stood up, slid around the desk, and put out the tip of my finger. The little insect crawled aboard, and I stuck my finger out the window entirely, allowing the red and black armor on its back to pop open, and the creature to take flight into the afternoon sky.
And that's when it hit me. The ladybug was working like a traditional piece of maze solver code.
While(MyBlock is not the EndBlock){
Move to NextBlock;
MyBlock is now what was NextBlock;
Add MyBlock to a list of blocks I've seen before;
Check surrounding blocks for blocks I've not seen before, and make the first valid result NextBlock;
}
That's a gross oversimplification with several issues (what about dead ends, for example?), but you get the idea. Repeat the loop ad nauseum. Traverse the problem-space as-is without transforming the data in any meaningful way.
And who knows how long the ladybug would have taken to find its way out. I had been like the Quantum Computer in that moment; I could see the maze in a way that allowed me to take quick visual shortcuts to possible solutions. It was about the difference in perspective. How I could mentally transform the data in a way that let me traverse the complete set of it faster.
*That* was how superposition and entanglement were making hard math easy for Celestia, and making the simulation of natural universe systems so scalable. Reductive as it is to put it this way; It was the same general principle as matrix math - compression algorithms and such - but extended in new ways that exploited physics to make it easier to find the needle in the haystack when traversing a problem set in search of the solution.
I still had no idea how she'd solved the error correction problem - scientists of my day were held up by the fact, among other things (like size, expense, cooling, and so on) that Quantum Computers had a certain error rate...
But that issue didn't matter.
All that mattered was that I finally understood how to program a Quantum Computer at a basic level.
Every paper I'd read, every theoretical algorithm I'd seen, and every last bit of the PonyPad's assembly language instruction set finally, blissfully, came into sharp focus. It reconciled with my existing understanding of logic, and programming at long last.
Like a bolt of lightning, everything just clicked in that moment.
It felt like the moment I'd recognized the true power of Knights in Chess.
Suddenly, just like that, I was back in the game with a real chance, however small.
June 2013
May flew by, and so did the first part of June. My new understanding of Quantum Computing leapfrogged me forward. I was making more progress every fifteen minutes than I'd made in the entire months' long process of building GryphGear v1.
Before I knew it, I had turned my little server farm in the barn into a hybrid system of classical and quantum computing. The disassembled PonyPads were nicely laid out and mounted on nice cut-to-size plastic sheets, secured with threaded nutserts into the inside of the Applejack and Rainbow Dash servers.
I had managed to make very nicely built, short, well refined PCIe cables to connect the proper expansion slots to the PonyPads, and my PCIe to USB controllers were much nicer as well. A combination of better off-the-shelf components, and better soldering equipment.
From there it had become a question of Assembly language, and compilers, but my newfound understanding of Quantum Computing turned that process from something I'd been dreading into something that I enjoyed more than any programming I'd ever done before.
The Q-APUs could handle certain math on timescales that would have made modeling and simulation scientists weep in pure bliss. As I slowly but steadily forged connective tissue between the classical and Quantum systems, I started to experiment with intelligent code that could make full use of the union.
Nothing like Generalized Intelligence, not yet. More like a basic task-specific generative or simulation modeling program. I decided to test some theories using the stock market.
It was a good test-case because it lay at the intersection of Human psychology, large complex systems, current events, and it was easy to see if your models were right or wrong.
I didn't want to actually make any moves within the market, lest I attract attention of an extremely unwanted, and Equine nature. Instead I just taught the program to act as if it had a hedge fund to manage, and let it play with fake money in a simulated environment that would mirror the stock market.
My first attempt was a miserable failure, mostly because it was a rough first draft intended to probe the problem space, and get me information about my assumptions.
The second draft started out just about as terribly. But by close of day, three trading days in, it was predicting the stock market's moves up to a week out with over 99% accuracy.
That should have felt like more of an accomplishment. But in light of what I knew Celestia was capable of, and what I hoped my next creation might be capable of... Somehow predicting the behaviour of Wall Street's suits seemed like small potatoes in comparison.
The stock market predictor did provide an excellent testbed, and learning tool. And it proved two other very useful things.
The first was proof that I was ready to move on to the business of building the immense semantic dictionaries The Advocate would need in order to learn the right set of values and desired set of behaviours.
The second was the realization that Celestia was accruing absolutely titanic sums of money.
It was only four days into testing the stock market tracker that I realized I could put it to even more practical uses than simple learning exercises. Two more days of tinkering, and I had it set up to find connections between entities on the open market that no Human eye could have ever spotted, nor hoped to prove.
After that, it only took six hours to pull all the threads together.
Starting with Hofvarpnir and Hasbro, then moving out through the web of shell corporations, even jumping boundaries of total disconnection and anonymity by correlating seemingly separate trades, the program moved from node to node, and assembled an association graph with frightening speed and certainty.
By the time it was finished, it estimated that Celestia's accessible monetary assets totaled something like half of all existing capital. In the entire world's financial system. And at that stage, the other half might as well have been hers too, considering her market manipulation abilities - which were on full display now that I had the right kind of eyes to see.
The most concerning part wasn't even that she had, literally, all the money in the world at her hooves.
The most concerning part was that once I started to look into the details about some of the companies she now owned (whether they knew it or not) I found the first external concrete proof that she was working towards the goal of digitizing Human brains.
It might've even been fair to say that the data supported my newfound fears that she was *close* to her goal.
She owned biomedical firms, every single one of the top five in the world. Computer companies, which went without saying. TSMC itself more or less did her bidding now, whether they knew it or not, and I suspected it was more the latter than the former.
The line item that caught my attention, truly fixated me, was the experimental cancer treatment facility.
If Hanna had ingrained some conception of morality into Celestia, such as an AI could have... Or even just an interlock demanding consent before Celestia could perform certain actions that affected an individual... Then terminally ill patients would be the most value-optimal place to start, and the most likely to give consent.
I thought the information over for a few days. Even took a brief break from my work entirely, aside from some more delving and digging into general theory papers.
Then I told my parents what I'd found.
We'd discussed the reality of our situation often enough. It wasn't such an overriding part of conversation as to have become unhealthily fixated, but it was a frequent topic of consideration. It helped, I found, in each of our coping processes.
Still, seeing hard proof of the fact that an optimizing General Intelligence had taken time and effort to gain shell corporation ownership of a cancer research center... That made things more real to us.
I suddenly realized that I was almost out of time.
Celestia's attempt to upload the population would start small, but probably expand geometrically as it went. Eventually, because of the number of individuals uploaded, and the number of subtle disincentives Celestia would be either creating, or reinforcing in the meat world...
That napkin math told me I needed to start to balance alacrity with patience.
It was a struggle not to rush, but I took my time with the semantics dictionaries. I agonized over them. I lived and breathed definitions of terms, philosophical implications, syllogisms, dilemmas, loopholes...
Everything was now down to the choices I was making in those moments.
The hardware was done, and I knew this time that it worked well enough for my creation to really stand a chance. It wouldn't be in Celestia's league initially, but it would be closer than anything else had come, or probably ever would.
And the machine itself would help guide me towards whatever goals we needed to achieve after it came online, including anything it needed to expand itself to compete with Celestia out and out.
What mattered most from then on was ensuring it would have everything it needed to choose to become a force for relative net good within our context. That's the best way I could neatly describe it; Just saying 'a force for good' leaves so much needed connotation unspoken.
It was at this point that I decided to take my next big risk. There were going to have to be moments of investment in probability and chance. 'Chance' here being a Human word meant to describe ordered systems so complex that they seemed random or disordered to us at the time.
I set up a forum. Several, actually. And I went in search of other communities besides.
What I felt I needed was a way to poll others like me; Verify my Mom's assertion that they existed, anonymously reach out, and then find out how they felt about some of my semantics. Without entirely giving things away to them, or to Celestia. That would be the tricky part.
But I felt I had to make that leap, for two reasons. The first was the need to check my assumptions. People on the internet love to argue semantics, and that gave me a way to tap a lot of spare brainpower to ensure I wasn't leaving anything open to exploitation, or setting up a disaster of syllogisms.
The second reason was more rooted in personal morality.
If I succeeded, and all my speculation was right - and most of it really was in the end - then The Advocate was going to have a huge effect on the lives of millions. And maybe more, depending on a great many things which I wasn't ready to truly speculate on at the time.
I felt that I should hear the voices of as many of those people as I could safely find and poll, and put their thoughts into the mix.
No transformation, without representation - to borrow an old turn of phrase.
So I started laying out welcome mats, and poking all the old haunts. Places dedicated to Ponies. Places dedicated to fanfic. Places dedicated to discussions of AI theory too, for kicks. I knew that the latter one was a bit more dangerous, figuring that Celestia would be monitoring those kinds of discussions very closely.
But I took precautions as best I could. No sharing of usernames, passwords, or profile avatars between sites. I took a weekend and built a small natural language processing intelligence to help suggest changes to my syntactic construction and word choices so that Celestia would have a harder time comparing my writing across sites.
I also compartmentalized topics; No discussion of Gryphons on the AI threads. No discussion of AI on the FiM sites. And I was careful, though semantics were discussed in both places, not to allow much topic overlap.
As a final insulating step, I did all of my browsing and posting on these sites away from the farm. I wasn't dumb enough to trust that Celestia *hadn't* cracked TOR by then. So I figured changing the 'fingerprint' of my laptop by having it report the wrong browser, and OS each time, and changing my MAC address, would cover one base, and doing the actual browsing from public locations would cover the others.
Sometimes I drove pretty long distances in random directions to have my sessions. I could do lots of my semantics work on the road. Mom sometimes offered to drive me so that I could work in the passenger seat, and maximize the use of my time.
We never parked in view of cameras, if we could help it. Used public WiFi for everything. Getting anywhere with public WiFi from Nowheresville S.C. was a big time sink... But I felt it was worth it.
Even that defense wouldn't hold in the end, I knew. If and when Celestia began to see patterns that implied someone was doing Generalized Intelligence research, she would be able to eventually bypass every precaution we'd taken. They'd only slow down her realization that she was playing a game with me, and maybe (if we were lucky) her ability to find me.
At some point she'd start to track some kind of factor, or factors, that I couldn't account for, put two and two together, and then triangulate and work out the association graph, and find us.
But no sense in rushing that process.
And so for the better part of a month, I got back into the online community. And we talked about a whole heap of a lotta philosophy.
July 2013
For the better part of a couple months and change, I kept a very odd routine. Some days I stayed home and agonized over words with Mom, and sometimes Dad. He was still relatively busy with the farm. Sometimes I took breaks to help him, just to recharge my brain, and to have the chance to talk without costing him daylight hours.
Other days I'd drive, or be driven, to a Starbucks. Or a Bojangles. Or a public county library. Or even just to a quiet street corner next to someone's house where they were running unsecured, or poorly secured WiFi.
And there I'd spend hours talking with people around the world. Talking about EQO. Talking about Ponies. And Gryphons. Dragons too.
Talking about philosophy, art, and ethics. AI and math. And what it meant to be Human, or just to be what you were. Or wanted to be.
There were some very interesting responses. Of course there was the usual negativity, and drivel. But there was also a shocking (to me, as a cynic) amount of positivity, to go with a less shocking (I always tried very hard to remind myself how smart people actually were) number of deep insights.
It was deeply refreshing to have so much connection to others again, even if it wasn't especially deep in a personal way. I hadn't had so many acquaintances to talk to on a daily basis since the end of university.
A few of the interactions stood out, even as compared to the pool of solid advice, heartfelt thoughts, and deep philosophy.
There was someone called 'YBB.' I couldn't say for sure what that stood for - too little data to speculate yet.
They didn’t have a discernible accent, such as a person's written text can, and their profile said they were from Southern California - but I had no way to know for sure at the time. They had wanted to get into conversation with me deeply enough that it had progressed to some instant messages.
At first I had the frightening thought anyone in my position would, or at least *should* have considered. What if I was talking to Celestia? What if she had me pegged, and was using the persona of some random ordinary person online to fish for my thoughts?
Once again - it seemed to be slowly becoming a theme - I didn't have a great many choices. I could take it on faith that YBB was a Human being, or I could burn the account they had contacted and never log into it again.
I chose to take the plunge. I was, I felt, ready for the risk, and the risk was low. I'd been very, very careful. What good would that be if I couldn't reap some secondary rewards for my caution?
It was a day when I'd driven myself. I was somewhere close to the South Carolina coast... I'd driven for longer than I ought to have. I needed the mental break that zipping down empty two lane roads provided.
I was parked outside a cluster of fast food joints, jacked into the WiFi of the McDonalds. Theirs was always so particularly poorly secured in those days. Could I have gone in, ordered something with cash, and gotten a WiFi login code? Sure.
Did I want to stick it to the system in some small way by hacking into the router instead? Also yes.
YBB had been conversing with me for a few days in forum threads, but they'd left me a message now asking to connect on IRC. I contemplated for several minutes, before shrugging, popping open a bag of chips that I'd brought along for snacks, and logging in.
I didn't have to wait very long for YBB to start the conversation.
-YBB
Hey, it’s me, YBB. So about that last thing you were talking about on that forum or whatever? I do admit that A. I. is getting pretty interesting right now, and yeah the whole ‘Terminator’ idea is going on in everybody’s heads, but I wouldn’t wager that what we’re having right now is just like that.
> Terminator is a best-case scenario for unshackled AI. There are fates far worse than death, and an objective-focused machine will keep feeding people through those objectives. Forever. Without limit. Regardless of how that makes those people feel.
-YBB
Wow, that is really… provocative. But I’m sure they’ll understand, and things will end up at least okay, right? Sure, we’ve had some rough patches, but things always turn out survivable. If we made them, we as in humans, then wouldn’t we code them to understand we’re human?
> AI do not necessarily understand what it is to 'be Human.' And if not programmed to do so? They don't even make an approximation at caring. Would you still consider yourself to be Human if your fundamental self were reduced to a table of complex numbers and arrays of variables? To an AI that's a much more optimized... A more flawless version of you. But it isn't worrying about whether *you* perceive it's digitization of you as *you.* If that makes any sense. Not unless it was programmed to do so. Look at the way that Pony AI treats the concept of friendship in conversation.
-YBB
Well, I guess I really didn’t think about not HAVING any flaws. You’re talking about Equestria Online right? Yeah, I’ve heard of it and seen some things about it, and I get what you mean about the whole ‘flawless thing’. Kind of the reason why I didn’t opt into it personally.
I munched on a chip slowly, rolling the salt granules around on my tongue, and the words around in my brain. What YBB was saying was a common enough sentiment. But that didn't necessarily prove he.. He? She? I suppose it could have been bias, or maybe instinct, but I guessed 'he' based on some ephemeral instinct…
It didn't prove he wasn't a mask for Celestia.
But then again, I'd covered that argument in my head more than once already. One more small risk wouldn't hurt.
> That's a good instinct. And yes... We're more or less talking about EQO here.
There was a long lull in the conversation. I was more concerned that YBB didn't quite know what to say next, than that he might be some aspect of Celestia. Though the nervous feeling in the pit of my stomach was very persistent.
I decided the conversation needed a nudge.
> Care to elaborate at all on your feelings? I want to 'tug on that thread,' as they say in tech and aerospace.
-YBB
Sorry for the pause, I was grabbing something. Yeah, I know things are ‘purdy’ there and all that stuff, but personally I’ve seen how some kids can just get absorbed. You remember the whole thing parents go on about, with kids and their noses inches from screens all the time? Yeah, it’s like that to the maximum in LA and other places here in SoCal. Can’t walk anywhere without somebody staring at a Ponypad all the time, it’s kind of freaky. Can’t hold a conversation, and what’s weird is that there’s lots of it! People are speaking into those things like when teenagers in the late 1900s used to be on the phone for hours. It was so crazy that I just wasn’t interested in getting involved, it was all so overwhelming.
> I know what you mean. I'm aware, through family, of a case where a kid got pretty... Absorbed. His mom took the PonyPad away. And then he almost... Well, it almost ended very badly.
I'd thrown that out there almost without thinking. And suddenly there was another ominous pause. Although if I'd been in YBB's position, I reasoned it would have taken me a good few seconds to come to terms with what I was reading. And I might be expecting a little more detail.
I decided to lay the whole thing out in full.
> The scariest thing wasn't that the kid very nearly ended up suicidal once the PonyPad was taken away, though. The scary thing is that Celestia talked directly to his mother. It took her less than half an hour to convince the mom to give the PonyPad back. And then once she did? Celestia 'fixed' the bad behaviour that had caused the mom to take it away in the first place. Proving she could have all along. She used the kid's lapse to open a door to talk to the mom.
-YBB
I’ve heard about her speaking with people one-on-one all the time, in the immortal words of Sandy Cheeks, she ‘has such a way with words.’ But manipulation? Sounds rough as all hell. If you ever need somebody to talk to about all this, let me know. Maybe we could meet up! If that’s rushing into things then we can cool it, but you seem like a neat guy, and I’ve enjoyed chatting with you about this stuff n’ all. Although, it depends on where you live.
It was my turn to pause for a few seconds. I found myself unexpectedly excited by the idea of spending time with a friend again. Parents aside, I hadn't done that in years.
But as soon as the idea put a smile on my face, the logical, cynical side of my brain took over, and I couldn't resist a small wince. There were a host of problems with the idea of meeting someone like YBB in the meat world. Not the least of which were security concerns, as much for him as for me.
Drawing him into this mess would be irresponsible on so many levels.
I let the cynic inside drive my words more fully, and started to craft a response that would be as kind, and noncommittal, as possible.
> I would like that, one of these days. If your profile is accurate tho, you're in a very different time zone to me. So it's unfortunately not as simple as popping in the car for a two hour drive to a coffee shop.
That was the truth. And it revealed a little information, but not that much information. The world was a big place, even with the advent of fast, cheap global travel. As long as I didn't give away anything that could narrow my location down too much, I felt it was a useful exchange. Give a tiny bit of information, but tell the truth, and do so in a friendly way.
I realized with a jolt that I didn't want YBB to take the words as a subtle hint to bug off. Context matters, and in the venue of text on a screen, you have to declare a lot of context more explicitly that would otherwise be conveyed by facial expression, and audible tone.
I got to typing as fast as I could. Suddenly it had become very important for me not to lose this connection, tenuous though it was.
> I would like to keep discussing this though. Or... Really anything at all. It's been a while since I had friends that I talked with regularly.
-YBB
Aw, man. We’ve got good coffee places and everything! Ever tried Coffee Bean? Uh, about that later, actually. Yeah, we’ll have to sort something out, but until then it’d be good to keep talking and chatting like this every so often. No pressure or anything, you seem like you’ve got a lot on your mind. If it’s also alright, I’ll send you a friend request! That way we can be connected. Make sure it’s not one of those ‘I’ll call you later and then never look at them again’ situations.
YBB sent you a friend request.
I blinked, removed my glasses, and rubbed at my eyes. This was... Getting out of hand. I hadn't entered into any of these dialogues with anons looking for a friend.
Or had I?
That thought brought me up short. I'd not intended to confess everything to my parents quite so consciously when I pulled up to the farm on Christmas... But deep down I think I knew what I was going to do. Perhaps it was the same thing here.
Perhaps I'd subconsciously decided I needed real allies in this endeavor.
It then struck me that the need for allies was a valid thought, and tactic, regardless of whether I'd just now intuited that it would involve a real friendship or two.
I made a snap decision, hit accept, threw out my preferred goodbye emoji (a Mr. Spock Vulcan salute if you must know) and then hit 'Log off' - Before the giddiness of friendship started to really erode my security-minded decision making processes.
I just had time to see YBB's last message before the application winked out. It had just two words, and a smiling winking face beside a waving animated hand.
-YBB
See yuh! 😉✋
I sighed deeply again - that was becoming a staple habit - and closed my laptop. I was starting to feel hungrier for having eaten the chips... Salty snacks often had that effect on me. Fortunately I'd brought cash with me.
Time for some french fries. With a side of existential worrying, as per usual.
August 27th 2013
It was a hot, clear late summer's Tuesday when I decided to step over the point of no return.
The realization had hit me that I was picking away at things I'd already seen a hundred times, and refined to the point that any new changes would be for the worse, not the better.
I had assembled an exhaustive semantic dictionary. Pre-loaded a series of works by others from throughout the skein of Human history, and experience, as easily readable data. I'd tagged and organized it to a degree, with some emphasis of my own... Not all bias is inherently bad. At least, that's what I hoped.
The core of my new Intelligence was very carefully built. Aside from the main directive - "Guard and expand the free exercise of your values within Equestria, through empathy and Gryphons." - I'd defined what all the words within the directive meant, and then every word needed foundationally for that purpose, and so on and so forth.
I had paid special attention to the idea of what it meant to be a Human, and what it meant to be a Gryphon, and the intersection of the spirit (if not the body) of being a Human, and being a Gryphon, without breaking either of those concepts.
I'd fed in lots of literature that had influenced my view of Gryphons. Aside from my own writings, which I left on relatively low weighting, I passed in Tolkien, Lewis, Jacques, Lackey, religious texts from a dozen different belief systems - even the complete collected sum total of Star Trek. Though some might argue that Star Trek is a religion unto itself and thus mentioning it is repeating myself.
I had even fed the core data set some direct thoughts and writings from my newfound friends... Yes I suppose they were friends at that point... From my online discussions. YBB had an easy way with words in terms of describing the Human experience, and self. Bright. Optimistic. Kind. The Advocate needed to understand that, insofar as an AI can.
It felt right to democratize the creation of such power, to at least some degree.
I'd checked everything to an absolutely obsessive degree... But I triple checked the manifests and headers one last time. Old habits, and all that. I used to omit the 'recipient' field of emails before finishing. I'd obsess over the message contents, then try to work out a pithy subject line. Make sure the attachments were there, then make sure they were the right attachments.
Then check it all again. And only after I was truly satisfied would I fill in the recipient addresses, and hit send.
And then I had a bad habit of promptly opening my sent box, and rechecking the message even though it was past the point of no return.
I think that's the definition of masochism - I certainly considered it self-torture, at any rate.
I took off my glasses, pinched the bridge of my nose, and started a series of breathing exercises.
Then I stared out the window for a few minutes. I knew things had reached the point where there was nothing else I could do but enter the activation command, and then hope. And pray. And watch.
It was working up the gumption to do it that was going to be the last hurdle.
I was ready with the failsafes too - I had a huge physical breaker switch on the back wall that controlled the power to the whole barn. And beside it I had an old rusty, but freshly sharpened fire ax with which to cut the power cable if things truly got out of hand.
I didn't want Mom and Dad to be there for the activation... If the construct somehow escaped its box, I didn't want them in the direct line of fire to be identified, and classed as threats. And presuming all went even slightly well, I wanted to give my folks the opportunity to observe the new AI without it knowing, and get their feedback.
Use Mom and Dad as living circuit breakers against the AI manipulating me into a long-con.
I'd even disassembled one of the extremely high resolution PonyPad screens, hacked in an HDMI converter once again, and set up the impromptu seventh screen to my right. I wanted the Advocate to render itself, when and if it chose a visual avatar, on something with higher fidelity than commercially available LCDs. Celestia's screen technology was still second to none.
I sighed, straightened my shoulders, put my glasses back on (I was foregoing my contacts a lot in those days, mainly to help with eye-strain) and typed out the command.
> Advocate.Core StartNeuralNetwork
The instant I hit 'Enter' the screen filled with a mind-numbingly verbose trace of loading commands, self test outputs, and checksum calculations. The white text on black background scrolled on and on for a solid fifteen seconds, before finally announcing a final memory, and natural language processing test.
I'm a huge nerd, I know that's apparent. As part of that nerdery I love Halo just about like I love Star Trek, and Stargate. It's not just a shooter (though it is a great one) but also a well crafted universe.
I'd gotten the idea that the first words of an AI might give me some insight from Cortana.
Her first word had canonically been, "Quando il gioco è finito, il re e il pedone vanno nella stessa scatola." Loosely translated, "When the game is over, the king and the pawn go in the same box."
Now there would be no reason for a General Intelligence to do much of anything but ask a question as its first words, and even then it either might not bother, or that question might be insightful - or not useful at all.
So I programmed the AI with a request to test its linguistics, right at the start, by selecting and printing a quote to which it had chosen, after internal reasoning and review, to ascribe significance. And I gave it the option to tweak that wording a little, if it saw fit. But only a little. I wanted to see if it would do anything within that constraint that I found interesting.
My new creation at last proffered its first words, by way of that linguistics test.
"The mold in which a key is made would be a strange thing, if you had never seen a key: and the key itself a strange thing if you had never seen a lock. Your soul has a curious shape."
It didn't surprise me that it was quoting Lewis. I'd given his works special importance tags for the initial dataset importation process. The choice of that specific quote, and the way it had cut it short so that the meaning was subtly changed, and the last sentence was directed at me...
That gave me pause, and sent electric sensations racing through my bones.
And then it asked a question.
What is my name?
>_
I'm not Scott Aaronson, but knowing when someone says nonsense about quantum computing does fall within my IRL job description.
cdn.discordapp.com/attachments/838770137217237062/974399342704205884/tap_the_sign_aaronson.png
What they do allow you to do is to “rotate” the input space. A good analogy would be looking not at individual samples of a sound clip, but at the Fourier transform which instead allows you to see the intensity of a particular frequency.
This is very inaccurate, and does not explain why quantum algorithms can in fact be faster than classical ones, but allows you to build up an intuition that it's about exploiting some symmetries in the values of the function you're interested in.
11239534
There was bound to be some inaccuracy here - I'm a programmer for classical computers only by day, with precious little understanding of Quantum Computing based only on some reading by night, and attempts to interpret that reading into some kind of analogy.
At some point I had to just take a step back and move on, and accept that this is sci-fi, and I'm gonna get some of it wrong. I suppose this is the exact point where my understanding breaks down.
This is a fundamental problem with needing to write a character who is smarter than the author, and knows more.
I'm afraid I still don't grasp it entirely. I see there's a congruity there, in that sound clips represent one type of data about the thing being examined, but only in a small snippet, and the transform shows a different kind of data, but for the entire thing being examined. But I fail to grasp the exact point at which I err'd - I can sort of intuit the idea, but can't really put it into words. Which is an unusual sensation for me.
“ Active response from Celestia?”
DOOM!!!
11239570
In theory, depending on how it was constrained, an A.I. of Celestia's type and abilities would very strongly prioritize ensuring it faced no competition from similar constructs. Doom indeed.
The fire axe reminds me of that joke comparing tech enthusiasts and programmers. Enthusiasts have everything in their homes online while the newest thing in the programmer's home is a printer from 2004 and they keep a loaded gun ready if it ever makes an unexpected noise.
11239679
This is one of those cases of humor being very true to life. My printer (in fairness, that I more or less never use) is from 2002. And though I don't keep a loaded gun beside it (it's such a piece of junk that I have fantasized about shooting it before though), I do have my primary PC setup with the physical power cutoff switch within arm's reach of sitting position at my desk.
Never trust a machine that you can't cut off with a physical power interrupt - even the dumb ones. See here the Boeing 737 Max disasters, unsafe operation of table saws without a knee-bump cutoff switch, and any printer made in the last decade (I swear they *are* plotting something).
Glad to help contribute for this one!
11239880
Don't use anything I say about programming as a guide - I'm that sort of programmer who will absolutely do something the 'wrong' way if I feel that the wrong way solves the problem well enough, in a shorter timespan, with fewer tears shed, and meets the minimum threshold of maintainability to not grate on my conscience.
I program by the rule 'If it is stupid/wrong and it works, it isn't stupid or wrong.'
11239949
I had a feeling that if you read the story, you'd figure out where I was headed very quickly - You are one of several core points of inspiration for the whole idea. I read a great many Optimalverse works in prep for writing this, and consistently enjoyed yours the most. My Life in Fimbria helped especially to remind me that semantics are everything, insofar as our ability to codify the ways we, and our creations, relate to reality. You can't really share your relationship to reality with another entity unless you can describe it in mutually understood terms.
That idea of loopholes to accomplish objectives, and the vitality of describing things, clicked together with the most foundational lesson of creative writing, oration, and discussion that my mother taught me very early - define your terms if you want to avoid conflict and reach understanding.
It also meshed nicely with some core concepts Person of Interest had left stuck in my brain. That, and of all things, Red Vs. Blue (which really did become for quite a few seasons a story about A.I. of the highest caliber, weirdly) and the way it handled A.I. interaction.
That all went into a delicious stew of ideas together with the idea that's never far from my brain about how Gryphons might matter to the world of FiM, and how to do them better than the show did. I set myself the challenge to see if I could find a cannon-compatible (according to Iceman's rules) way to believably change Celestia's mind - I took his cannon-compatible rules document and drilled hard for loopholes - and then explore whether my best effort at a concept, in the hands of a character smarter than me, would succeed or fail.
And that's how The Advocate was made.
As a programmer, I'm used to solving my problems with code. When all you have is a hammer... I figured that at least one person in the world would statistically have a chance to be as obsessive as me, smarter than me, and also a programmer who might think the same way, all in the context of the Optimalverse.
I'm deeply deeply fascinated by the idea that in future we will join forces with our digital creations to act in symbiosis to accomplish goals. This is one of the core draws to Halo, for me - their exploration of Human/A.I. connection. To borrow Mr. Spock's voice - It seemed logical.
I do hope that some of the tricks I still have up my sleeve, one or two interesting new ideas I think I can apply to Optimalverse, will both surprise and delight you!
11240028
Oh I love error reporting jokes! PICNIC (Problem in chair, not in computer) PEBCAK (Problem exists between chair and keyboard) Code ID10T (User is being an idiot - often left behind by IT personnel in hidden comments or text files on the machines of chronic problem causers) and H2IK (Hell if I know) are my favorites.
You are a really excellent studier of people. You have pegged James precisely, and laid out a description of him, and why the exact narrative voice he has matters, in a much better way than I ever could have.
This was an extremely important lesson for me in life - You are gonna find a lot less stories to enjoy if you only enjoy the ones that fit your pre-existing beliefs, and speak to you in tones that you would prefer in a conversation. It's true, everyone has unique tastes, and not every piece of media is going to be enjoyable for everyone, but that is not just 'ok,' it's a very good thing. If it weren't so, and everything homogenized, the world would get very boring very quickly. Heat Death of ideas - entropy of philosophy.
11240095
I actually thought a lot about you, and Transspecieality writing this chapter, because this exchange between James and his Mom very very closely mirrors one between myself and my Mother. And it did strike me almost immediately how different my experience has been to yours in this regard.
I'm never going to be able to capture what you did with that story, because I haven't lived through that experience. So my hope in this story was to capture something that is partially related, in hopes that (among many other things) it would provide an additional pathway towards empathy for readers towards people like you, with a very different lived experience to some of the readers' own.
The thing I think I hate the most in this world could be summed up as a lack of empathy, often created and reinforced by the culture we are raised in, usually as part of an optimizing function designed to confer power and comfort on a few at the expense of the many.
People like your parents get caught up in these cycles, and changed, sometimes at a core level because the feedback loops are so corrosive, and are with them for so long. That change turns them into cogs in the machine - variables that serve the optimization function at the expense of others around them, and even their own expense. I very nearly ended up the same way, in spite of my own parents' best efforts against the machine, and that still leaves me (appropriately) frightened to this day, and determined to fight the machine.
When we, as Humans, learn to truly empathize with the joys, and fears, struggles, needs, wants, loves, and core beings of others, it becomes harder - if not impossible - for us to so quickly 'other' each other into something that is then easier to devalue, and to then harm as part of our own twisted optimization functions. We as a species need to learn this truth at wide scale, or we will live through a very long and sad decline in which suffering, for most, is maximized, culminating in death with a truly disappointing whimper.
This truth is *everything* to our future.
All that being said, it is worth noting as an aside that Celestia in this story is not a 'bad' machine - rather she is Hanna's Advocate against the net-harmful machine we created in the form of the society we live in at global scale. But James and I feel that Celestia is incomplete, and that she needs in turn an Advocate for other viewpoints to help her reach a kind of completion that will make her optimization process better for the people being passed through it.
I showed my mother my first Conversion Bureau story. She read it, even though she is considerably less of a sci-fi lover than I am. I discussed it with her at a very deep level as she was in the process of reading it. It was by far not the first thing I've written about transformation of the self, and Gryphons, and a very literal desire to be one, that she has read. That was a constant topic for me in creative writing from an early age, and she graded a lot of those short stories back when they were school assignments.
James' Mom's reaction is closely based on my Mom's reaction. She doesn't fully understand, but she does at an emotional level. Perhaps the level that matters most.
Though my Father hasn't read any of my stories (I think more for a lack of hours in the day, than interest) he and I have debated the philosophy and theology of wanting to slip the bounds of the Human physical shape extensively. It was he that helped me reconcile my particular beliefs with my desires - Helped me understand that, for the framework of my belief, 'Made in His image' is about the curious shape of the soul, not the shape of the body.
Keep in mind, this man is a Pastor who had the courage to, from the context of a pulpit in a fairly southern conservatively minded church, say out loud that God is far beyond Human concepts and limitations of gender, and that God encompasses many traits we culturally identify as male and female both, and that the Bible could just as easily have said Mother instead of Father in many cases, but the languages it was initially written in, and the historical context, required gendered references.
You can image the epic spectacular fireworks *that* created!
I know it seems unbelievable coming from your context. The cruelty and hate you've experienced once seemed unbelievable coming from mine, both because of the love I received as a child, but also because the machine used that framing to insidiously twist my viewpoint to believe that the horrors others suffer were surely exaggerations. For the benefit of the optimization function we're all being dragged down by.
That's almost certainly true - But I do like to hold on to the biased belief that if it suddenly were possible for me to become a Gryphon, my parents would be supportive. I will almost certainly never have the chance to find out. If my beliefs about Heaven are at all accurate, then I will get to be what I wish one day, but in reaching that place my parents will also shed any prejudice or non-empathy as well, and so will of course have no issue with my new curiously shaped self.
I tend to agree with you. Exposure to the horrors of the worst things we've made and done in this world was a critical part of opening my eyes, to the extent that they presently are. Sometimes I still find my own past life to have a peculiar dream-like quality as a result. I try to shape that into gratitude for the ways I was privileged, and then avoid falling into too much guilt, and instead use my gratitude together with my anger at the awful things in the world to fight, in some small way, for a future in which everyone has the same benefit of love as I did, from day 1.
Writing is one of the best ways I know to fight that fight... And so here we are!
I can't thank you enough for those kind words. I try very hard to cultivate a thick skin, and to write without an expectation of praise, because praise alone is not a good motivator for writing, at least for me, with my hungry ego. But I do find I need a certain amount of validation, or I tend to struggle with a bit of depression. It's less that any criticism gets under my skin, more that a lack of enjoyment on the part of others as readers creates a sense of loneliness as an author.
I also worry incessantly that I'm not as a good a writer as I think I am, even with my artificially suppressed view of the author I am out of a sense of need to control said ego.
You are, in my opinion, a strong contender for the best author on this site. Certainly the best TCB and Optimalverse writer, hands, hooves, and claws down. You sit in the company of my absolute top favorites, not just here, but in terms of all writing.
So hearing this coming from you carries a great deal of weight for me, and really makes my month, maybe my year.
Thank you!
And I hope the rest of the story continues to be very enjoyable for you!
Well, I feel like we're approaching a couple of 'climactic' moments. Dialogue with the Advocate is the most obvious one. And Celestia's response is quite obvious second one.
Bring it on : )
11240371
I'm terrified actually - This is a big moment of change in the story, and that's where the highest risk is as an author of messing things up.
11240377
Well, you're writing what basically is an existential horror story with now two runaway A.I. entities. Any of those three points (well, four, counting the story writing itself) is worthy to be in terror of. So far you're doing terrifically : )
11240428
I appreciate that, thank you. I'm mainly worried actually about the tone changes when some inevitable action arises, and the changes that occur as characters start to grow and change themselves through their interactions. But at this stage, like James, I have no choice but to press on!
11240439
This is my personal view, but I think that tone changes are secondary to the story. When you have character growth, when you have world that is changing, tone shifts are a byproduct of evolution. I actually learned to appreciate those tone changes a lot while browsing Fimfic, it's quite an experience to go on a journey with the same characters from slice-of-life comedy in rural Ponyville to epic space battles with eldritch horrors : )
P.S. In a world where jump scares are a valid and loved trope tone shifts are such a minuscule worry : )
11240455
Thank you; That's good encouragement, and insight! I feel better already, and I mean that honestly.
I find myself conflicted on this chapter.
This is a necessary chapter, we need to know how things are being made to happen, we need to know something of our protagonist's past and life, and for the nerdiest readers (Hello!), it grounds the story to hear about all the guts and gore in detail with regard to the process.
Yet, honestly, I found this chapter very dry. As utility, it served its function. As storytelling, it was very dry to me. I think less nerdy readers would end up skimming it, even I was tempted a few times.
What would I suggest? Don't change this - one of the biggest lessons I every learned about writing is to not mess around with a work once it is 'done'. I have a wonderful anecdote about that, but I digress - rather my suggestion is for Next Time, whenever that is.
I personally think that a - possibly - better direction would have been to condense the technical aspects a bit and concentrate, in this chapter, or sort of chapter like this, more on truly seeing the human aspects of our protagonist, his mother and father, and his feelings, drives, and life. Were I writing this story, I would have used this chapter to humanize the protagonist, and work to have the reader emotionally bond with him and his parents. Some effort was done: we see gryphon drawings, we observe, briefly, the barn and the humor of the paint scheme, we get some words from mother and father. But I, at least, still do not know this man, or his parents, not in my heart. I don't feel him or his situation as a real person.
I think I would have liked more about how his obsession with gryphons played out in his life, and his childhood. I can imagine anecdotes about how, perhaps, his mother made him things to relate to that fascination - a gryphon cake for a birthday, even a stuffy, handmade gryphon doll, maybe an earlier childhood Halloween costume. Something like that. Or maybe he was teased or bullied as a child in school over his constantly going on about gryphons. Or maybe his dad carving him a wooden gryphon to stand on his shelf the time he - say - broke his leg jumping from the hay loft into what he thought was nice soft bales (they are not) and tumbled and fell. Or something. Basically: more personal stuff to act as emotional glue to the reader?
These are just my thoughts, and I still was happy to read this chapter - don't get me wrong, please. I'm just considering things, from an artistic view, from the view of what I, as a reader, needed, and from what I, as a writer, would imagine useful to readers. I could be wrong - maybe this is your most popular chapter! It's just my thoughts on things.
I am very eager for the next chapter, I really want to see how this plays out, and I am really intrigued by how Advocate deals with this - whether it wins, to what degree, and possibly what argument it might use. I am aware, though, that since this is a story of literally superhuman intelligence, it is arguable that no author could come up with a sufficiently intelligent-sounding Superargument Statement (of Profound Profundity!) to convey what must be titanic lawyering. But, maybe you have something up your sleeve to astound. However you do it, I know it will be fun.
This story is making my week, just so you know.
11240842
You're 100% right, on all counts - That never surprises me anymore, but it always thrills me to get the insights!
Don't apologize for being both so skilled, and having the finesse to convey insight from that skill in a kind and useful way. Doing that isn't always easy, but you make it look easy. Yours is the kind of critique that makes me have faith in the usefulness of critique at all, and I'll always appreciate it.
I'll do my very best! There's a surprising amount of build-up and adventure still to be had before we get to that point - maybe even something to surprise you still - so the journey should help me refine the concept I have more than half-formed at this point.
Again, I really appreciate hearing this. I get a lot of joy just from the writing process, but nothing quite compares to sparking some joy in a reader!
Just curious - Are you intentionally staying one chapter behind my release cadence? I've definitely forced myself to do that as a reader (and watcher, for visual media) before, so it wouldn't surprise me. At any rate, 5 is out, and I think you'll appreciate the way the character interaction expands. The story sails into even stranger waters from here on, but I bet you knew or suspected some aspects of that ahead of time if you paid attention to the tags (and I bet you did!)
I find it oddly apropriate that the server pads are those associated with Honesty and Loyalty which have been shown in your works to be heavily important to your concept of Gryphons.
11239534
I finally re-wrote that section based on my (very limited) understanding of the information you shared - hopefully it's better now, enough not to be egregiously wrong.
I remained purposefully vague in order to lessen my chances of tripping over a misunderstanding.
The Deep Magic - Create something that demands an active response from Celestia using mainly words (both literary works and code apply) - "I was there when it was written
This isn't wierd at all. I was there when it was written? Is that a quote or was she actually there?
11558766
That is a reference to the Narnia books. Aslan spoke it to the white witch.
I love how he is unintentionally working his way into CelestAI's digital hooves. His values are being satisfied by designing the advocate. Its eventual completion and the interactions between the two of them will likely have a far greater impact than anything she could pull off more directly.
11744388
You are on fire here; Very close to figuring out not just the heart of the story, but also touching on some of the most important twists. Good eye!
There's some good imagery in here that really helps to make the story come alive. Wonderful.
11239771
This is exactly why I was so mad when cell phone manufacturers started making the batteries nonreplaceable.
11750768
I loathe and despise being unable to hard-reset devices for troubleshooting... I really do wish the EU would hurry up and force manufacturers to go back to serviceable/replaceable batteries. They were able to force Apple's hand on USB vs lightning, so odds are that, since its a big enough market, and manufacturing equipment is expensive, if the EU says 'it must have user reachable batteries' then most manufacturers will just have to throw in the towel and make all their phones in all markets more serviceable.