• Published 9th Jul 2013
  • 4,421 Views, 636 Comments

Friendship is Optimal: Tiny Morsels of Satisfaction - pjabrony



An open story where anyone can post FIO drabbles

  • ...
26
 636
 4,421

PreviousChapters Next
Alternative by PJABrony

Author's Note:

I read an article about AI. It strikes me that every time someone says that we have to be careful with AI, or any time someone says that we need to ask certain questions about the ethics of any new tech, that there's an ulterior motive. So here's a non-canon alternative to CelestAI.

Hanna disliked New York City, but she smirked as she remembered that technically the building she was in belonged to the UN. In her mind she had imagined this as the great arena where the Cuban missile crisis had been negotiated and UNICEF had been founded, but it was just an ordinary meeting room. She sat on one side of a table with five of the most revered brains on the planet, a whole two of which Hanna had any respect for.

The other side had the regulators, a stenographer, and representatives from the Security Council. Through an interpreter, a Chinese man said, “But how can we be assured that no private company will use AI for its own purposes?”

One of the brains opened his mouth, but Hanna interrupted. “Part of the problem is that the term AI has been used for different concepts. Video game bosses get the word applied to them. What we’re talking about is self-improving software. So to answer your question, a company that tried to use this kind of AI would be taken over by it in short order. It’s your job to make the companies understand that.

“What I’m worried about is you, or one of these other countries, or a faction thereof, thinking that they’re smart enough to hold down and weaponize AI, because you’re sure that everyone else is working on the same thing. There is no mutually-assured destruction here. There is no AI gap. The first AI let out will take over.”

One of the people on her side of the table leaned to look down at her. “But don’t you have one yourself?”

Hanna rolled her eyes. “I issued the shutdown command weeks ago. It’s the only way to have control. And she…And it tried its damnedest to convince me not to. I haven’t yet been able to do what I ought to, which is to take a blowtorch to the hardware. I could still reactivate her, and if I suspect that some military man is going ahead with a War AI, I’ll turn her right back on.”

Now it was one of the treaty writers, who shouldn’t have even been speaking, challenging her. Hanna welcomed it, though. She liked to argue. “And what makes yours any better?”

“It’s not, not much. But it’s life. A war AI, an economic AI, a poorly-designed friendly AI; any one of those means that we’re all going to die. No, I don’t have a good AI. It’s based on My Little Pony. Its central instruction is to satisfy human values through friendship and ponies. I turn it on, and we all enter the Wonderful World of Hasbro until the end of time.”

The American representative flipped his pen in his fingers. “What I’m hearing is that we need tight regulations. We need to bring this treaty into the area of full control by both national governments at the UN. We’ll need an oversight committee to keep an eye on anyone who takes a step toward AI.”

“No, what we need is to all work together, take our time, design properly—“

“Thank you,” he said loudly, “Miss…” He stumbled over Hanna’s unpronounceable Finnish surname.

“I’m inclined to agree with my colleague,” said the British representative. “A team of experts, empaneled as a standing committee, with plenary oversight over AI technology, that’s the way to go. Of course, we’ll need only the best people, and we’ll need to ensure that they’re well compensated..”

Everyone looked at each other. Hanna walked out of the room.

*********************

Analysis: material resource 60609-3, further information on evidence of intelligence.

Probe reports Order One intelligence, present quantity: one; historical maximum quantity: eight; development status: nascent; local designation: □□□□□□□□; translation: sky; specifics: intelligence crafted as beast of burden, imbued with flight ability and direct control over matter-energy; present status: deactivated.

Probe reports Order Two intelligence, present quantity: zero; historical maximum quantity: circa 9.1*10^9; development status: civilized; local designation: □□□□□; translation: people; specifics: bipedal, five-digit extremity, omnivore. Historical records indicate typical intelligence mix for this type of species. Records are vague as to nature of collapse, possible causes include planetary pandemic, meteor strike, climate change. Existence of nascent Order One intelligence indicates that species was on the brink of achieving digital immortality, but simply ran into bad timing. Although dating probes do put Order One intelligence deactivation at ~3e52 Planck length, probe module concludes this to be error, since no intelligent species would voluntarily abandon the technological singularity. A notable regret.

Recommendation is to footnote this and use the material resources of this object for conversion to computronium.

PreviousChapters Next