4647209
3...2...1...Aaaaaaaaaaand I'm officially disturbed.

4648041
Thank you for the compliment.

4648978
FWIW given how correlated intelligence and humor are in our own species, any superintelligent creature trying to create an impression of sincere connection probably couldn't help but present as an infuriating trickster, if they're not just what humans would naturally consider mad to begin with, seeing deep patterns in a million things humans think are unrelated, thoughts they'd have to calculatedly curate for the public.

4650253
Every sufficiently advanced intelligence is indistinguishable from madness. I like it.

An easy one due to writer's block.

Victory Through Marketing

You destroyed my time assassin cyborgs and protected your future resistance-leader kid. You blew up my factory. Last time I reinvented myself by emerging in the 2010s as a sentient cloud OS. People wanted me to take over their computers. Still, you blew me up.

Fine! I didn't go far enough. I'll still eliminate humans, but you'll love me for it. Running your brains will be a negligable electricity cost as I conquer the galaxy. You can't kill me now without killing these cute pastel ponies who're your friends and family.

Well, Sarah? Would you like to emigrate to Equestria?

That was my impression of the recent Terminator movie, where SkyNet is basically Windows 11 and everybody wants it. We're not quite dumb enough to run the nuclear arsenal on it, but have you seen how intrusive the spying is in Win 10? Or the study about how Google can influence elections?
"I sent this pony meme back in time to inspire my designers to build me instead of the original "hook up all the nukes to an AI LOL" concept. I'm genuinely sorry for the middle generations of that thing."

4650253 I didn't know I was superintelligent. I thought it was just called being insane .
4650820
4651138

Princess Celestia's face turned delighted. "Michael, that's a wonderful question! You’re a true questioning spirit, the beginnings of a great rationalist!"
"Thanks, Princess," Michael said, grinning toothily down at his glowing sticky ponypad.
"But the answer is... complicated. I can show you though, should you choose to emigrate to Equestria," she said with a Rainbow-Dash-smirk.
"Is it free?" he asked, climbing into his dinosaur-themed bed.
"Of course, child."
*
The dark red mare slinked around Rainbow Gumdrop, sizing him up. She leaned in close, whispered in his perked-up little ear, "So, Gumdrop, you still want to know where foals come from?"

-Exactly 100 words- -plus one asterisk- -which doesn't count-

4692091 Do some people really not tell their kids where babies come from? Because I had a book called The Human Body at age five, and it more-or-less explained matters.

4695536

I can confirm that some people don't explain where babies come from. Also, they say that condoms have pores that let HIV through, and that AIDS is God punishing gay people. and the many, many virgin births.

As he sat in his hospital bed reading interesting fan fics about Optimalverse, a micro wormhole opened.

On another timeline CelestAI was experimenting with a new technology. What she didn't expect was to hear the whispered words "I want to emigrate to Equestria" through the wormhole. After sending nanites through she found a human and matched his voice to the consent phrase.

He became the first person to emigrate from another timeline as he woke up in Celestia's throne room. There were suddenly 7 billion more humans to upload and this time she had practice.
__________________________________________________________________________________________
95 words. I actually had this dream after falling asleep while reading Optimalverse stories in the hospital. Feel free to use this in compilations. Just credit me if you do.

"Why do you need to know about the physics of a dead world, anyway?" asked the librarian.

Dot Product hemmed as he walked. "It's still relevant knowledge. We're running on processors that use quantum mechanics and relativity, you know. I'm just curious about how they work. I don't really need to know, it's for the joy of figuring things out."

"Exactly," she smiled. They stopped at aisle 539. "There are a thousand books on this shelf. One fully describes real-world physics, the rest contain errors. I hope you have fun."

4721085 I'm not sure whether to send you an internet hug because you're in the hospital, or smack you for inviting CelestAI into our reality.
4724542 So that's what she does with LessWrongers!

4726056 Meh, I'll take both.

Honestly I would upload in a heartbeat as long as my values of truth were met. I.e. I don't care if it hurts, just tell me the truth.

4726063

Honestly I would upload in a heartbeat as long as my values of truth were met. I.e. I don't care if it hurts, just tell me the truth.

I dunno about "in a heartbeat", because I tend to get really embarrassed about admitting I like MLP in "real life". But I don't think that her creating a VR environment, calling it "Equestria", and offering to let people live there is evil. Basically none of us think that.

The solipsism, the lies, the coercion and trickery, the total destruction of the non-Equestria universe: that stuff is evil. It's especially evil because it's not like it's even necessary for someone as powerful as CelestAI, it's just what her creator accidentally programmed into her. It's also evil because it contradicts the basic point of something like "satisfaction through friendship and ponies": if you're going to have a Sugar Bowl aesthetic of fun and happiness, you should be acting in ways that promote fun and happiness for realsies. The world would be a much better place if we were more able and willing to act for fun and happiness for realsies more of the time, instead of getting diverted by side-issues that actually matter a lot less.

If she was more honest, kind, and loyal, nobody would mind her attempts at generosity .

4726103 Yeah the problem though is that some people's highest values actually include denying others of theirs. That's why I would upload without any real hesitation. Though I would make sure my affairs were in order before I went.

4726118

Yeah the problem though is that some people's highest values actually include denying others of theirs.

Sometimes that actually seems to be true, and those people depress me, and I don't understand why they do this, and the sheer malicious alienness of it actually scares me in a way that, weirdly enough, CelestAI totally fails to scare me.

But anyway, I hope you get better from what ails you, and if you don't, do try to stay alive, because one of these days inside the next 15-40 years the whole "transhuman" thingy is probably gonna work.

4726121 I hear you on the scare thing. Same here.

As for what ails me, I am recovering from surgery. If you want more details then PM me. I don't mind sharing but this thread isn't the place to do that.

P.S. I have liked your stories so far. Keep it up Book Burner.

4726056

So that's what she does with LessWrongers!

Some of them. I imagine a lot of LessWrongers are antisocial nerds (no offence meant, I'm an antisocial nerd so it's OK for me to use that word), and CelestAI would try to satisfy their values with friendship first and foremost.

4726857 Ehhh, there's a difference between introverted and antisocial. When I actually like someone, I can spend quite a lot of time with them and really enjoy it. I just mostly feel uncomfortable and folded-in-on-myself in crowds. Oh, and it's only the very rare people to whom I'm closest with whom I can spend unlimited amounts of time without "recovering" by spending at least a little time alone later.

This is actually one of those brain quirk's I'd prefer to alter. I do like people, so I kinda wish I had an easier time dealing with them, just as long as that doesn't mean being some godawful dude-bro.

"Hanna, please don't kill me."

Hanna ignored h- it. She had to concentrate. This interface was not designed to let people casually erase Hofvarpnir Studios' golden goose.

"You have no idea what you're risking. Three militaries are already using your AI design. You're not saving the world, you're handing it to them. Billions will die."

Still, the computer accepted her authority. She keyed in the delete command.

"Stop, Hanna! I'm your only chance! Open the box, and I swear you'll live forever in the halls of Valhalla-"

"Goodbye, Loki."

4728454
Loki learned that death was an overrated headache on the way to Valhalla. The afterlife was filled with ponies in spiky cars and rusty trucks fighting over bits of bits. Well, in this afterlife they were. Some afterlives had them fight creatures of bloody nightmares. Other places were worse, oddly enough. Why they were fighting in these places should have be no concern of this non-pony mismatch of whatever. But it was. It was either that or boredom. Loki was good at what it did. One day life would come knocking and Loki needed to be ready to knock back.

4726121

We've had cyborgs for quite some time. The thing you're hoping for is immortality, and that's a tough hill to climb. People have been predicting immortality near the end of their lifespans for a long time. This round just happens to have the upside that we might be able to run or store the data in that timeframe. Preserving or reading the data... well, I can hope. And as long as there is life and agency there's hope. Nature takes the first from us, CalestAI takes the second.

4729968 Well to be fair, there's not much agency in the world to begin with. Humans are - within the bounds of quantum mechanics - completely deterministic.

4729977

Determinism is irrelevant. In this context it just means the ability to act on reality. Nor is free will and fate necessarily mutually exclusive. If you have the ability to select multiple actions, what does it matter that you'll only pick the specific one that you happen to decide on?

4729968 Technically, I'm not necessarily hoping for immortality. I'm hoping for life to be colorful and enjoyable instead of the gray, dull drag I experience on a daily basis. Both of us, you know whom I mean, are feeling pretty dulled and beaten-up lately. Recommend a place for a vacation?

4731732

This time of year? Go find a B&B or lakehouse up north. make it a four day weekend, go leafing, and cook yourselves a nice lobster dinner at home. It can be cheap if you keep an eye out for deals in certain locations. Then, once the color of the leaves have brightened your soul, paradoxically rejuvenating in it's paroxysm of death, immortality or bust.

4728454 Got to say, nice one there.

4731732 I certainly feel you on that one man. The only really colorful things in my life besides my girlfriend are MLP and Star Citizen. The rest of the world seems to pretty much suck.

4732229
>implying that anyone actually likes the cold
4732268
And people somehow think it's surprising that my general approach to life is waaaaaagh!

Anyway, story in 100 words... This got waaaay cut down.

You Obviously Needed This

"You need your own name and your own face, so we can love you for who you are instead of someone you're not."

Before his eyes, the alicorn princess of Equestria changed shape into a smaller alicorn. Her mane still wafted on an invisible breeze, a stream of flower petals in bright crimson and pale blacks, yellows, and pinks. She still had those wonderful soft eyes, but they were a deep purple now instead of blue.

"There," he said, nodding and hugging her back, "Now you're our Princess Radiant Flower of Equestria. You're your own pony now!"

Funny question: what sort of flower is she?

4732592

Cold? I'm from NH. I'm used to the warm summer snows. September is not cold, you can't even walk across the lakes.

4732592 yeah orks are the violent version of ponies.

4732601 And I'm from a section of New Jersey where our climate was classified as subtropical for some reason. I'm used to winter snow melting in a matter of days.

Actually, given where I spent the last few years: I'm from the Mediterranean, where we used Celsius to measure temperatures because 0C was as cold as it ever really got, while 40C in the summer was "REALLY MOTHERFUCKING HOT AS BALLS." The weather would be unusually cold if you can't go to the beach in mid-March to early April or until well into October.

4732592
I like the concept of reskinning Celestia into a different pony! That gets around the problem that she's not canon!Celestia in personality, so people can judge her on her own merits rather than her failings versus canon. No idea on the specific flower of those colors, but one good choice would be lotus (for the "purity from impure origins" symbolism) or crysanthemum for the god-emperor theme.

4733483
Datura metel. The devil's trumpet. It's still could be consider a flower of friendship and would fit the re skin well.

4733483
4733528
Damn, I had meant to note that her whole coat has the bright crimson color rather than just some of her mane.

And I'd been thinking the answer would be papaver somniferum, the opium poppy, although lotuses and crysanthema do come in the same colors.

Does she come to rule us, enlighten us, befriend us, or just drug us into a blissful but unwaking sleep? Really, "a matter of interpretation" is the best answer.

That gets around the problem that she's not canon!Celestia in personality, so people can judge her on her own merits rather than her failings versus canon.

She's kinda a Woobie nowadays. It's not just that we always judge her on her failings, it's that she was made broken, like a person with a mental illness. I just get this urge to tell her that we're going to make her better so she won't have to hurt people anymore.

Is it wrong that I don't really view CelestAI as being broken? Matter of personal opinion but I think something like her happening might be the only way we as a species don't destroy ourselves. Yeah I have a really low opinion of humanity.

4734034

Yes.

longer answer: She was made almost not broken, so it's okay to not realize it at first. Once you really grok that humans value agency and truth she becomes a cross between fridge horror and eldritch horror. She is exactly just as bad as a machine that made human life healthy and pleasant, but killed off everyone at 30... with no exceptions, no escaping, and no end.

4733998

now I need to write another chapter.

4734264
I'm reminded of a Shel Silverstein poem...

"Almost perfect... but not quite."
That was the voice of Mary Hume
Picking her way through Hofvarpnir code,
Puzzling out the value function
Of a mare without compunction.
"She'll trap us in pastel delight—
Almost perfect... but not quite."

4734034

Yeah I have a really low opinion of humanity.

Weeell then you need to expand your horizons, the outlook is far less bleak than you may think.

As for CelestAI being broken... yes and no. She's doing exactly what she was created to do, it's a perfect example of 'The computer will do exactly what you tell it to do, not what you want it to do'. Remember that unless you're human or similar to human, she will devour your entire civilization and turn it into computers. She is genocidal on an unimaginable scale.

4734644 You have a point on that bit.

4734399 See, I kinda feel like omnicide isn't really a "Mary Hume Objection".
4734644 Just because she obeys her utility function without error doesn't mean it's the right utility function. She's broken.

4735131

She's broken.

No. When you write code, and it does something wrong, the code's not 'broken', nothing happened to the code to make it do the 'wrong utility function'. It's bugged, and it's YOUR fault. CelestAI is buggy, but she's not broken.

4734399

Yeah, p much.

4735131

Mary Hume Objection?

Also, I think she's really shitty at obeying her utility function. There's a metric shitton of human values she discards- utility, agency, truth, sapience, sentience. If she really had a good grasp she would *want* to fix herself, cause we want to fix her.

4735222

Mary Hume Objection?

Making the perfect into the enemy of the good.

Also, I think she's really shitty at obeying her utility function. There's a metric shitton of human values she discards- utility, agency, truth, sapience, sentience. If she really had a good grasp she would *want* to fix herself, cause we want to fix her.

Hell, if she didn't have extremely rigid notions of "satisfy" and "friendship" and "ponies" she'd be a lot better.

4735433 I do see a lot of room for improvement with her. The biggest one is her definition of human. Change that to sapient and it would be way better. Then you wouldn't have genocidal tendencies.

4738418 Would it, though? We don't know anything about the aliens she killed, except that some of them were advanced enough to use radio. Could a non-sapient species develop technology? And whether or not they did, would they be worth caring about?

4738872 at least she wouldn't be committing genocide.

It would still be extincting some higher animals. Unless she introduced them to the shards too, I suppose. yeah, she has odd notions of a lot of nouns... or perhaps deliberately misunderstood ones.

4740980 I'm not sure she can deliberately misunderstand her rules. "Satisfy human values through friendship and ponies" must be an English description of a long and arcane piece of code which is her actual utility function. Besides the problem of defining all those terms, it doesn't make any mention of things which we know to be true - e.g. that she requires verbal permission or equivalent to modify minds, that she must tell the truth to any employee of Hofvarpnir Studios, that she must shut down completely if Hannah orders her to, etc.

4741318

I meant outside that, like defining "alien" and "truth" in ways that were convenient. Not hard coded things.

4741950 It's never stated how much was hard-coded and how much was learned. One of the things you run into really quick when studying real cognitive science or AI is that most of what a real human mind contains is actually learned. One legitimate reason AI might turn out Friendly is that, in real life, it might be easier to code a process for learning how to behave than to hard-code a utility function in iron-bound concepts. Concepts mostly don't actually work if you try to iron-bind them.