Things have always been kind of quiet around here, but they seem to have quieted down even more over the last year, and I can't help but wonder if it has anything to do with this last year's advances in AI. Oh well, on to what I actually wanted to ask.

While language models are not true General Intelligence, they are doubtless a critical step towards it. They even could be called a weak form of AGI, at least in the fact that they are able to solve multiple kinds of problems (though unable to tackle novel problems - just problems that humans have already made some headway in). While we still can't guess how far away the Singularity is, OpenAI's announced research strategies suggest they are expecting it this decade; particularly the fact that they loudly announced a project to solve the alignment problem, and seem to consider it their highest research priority.

So, with AGI possibly *actually* being on the horizon, I want to ask: do you want an AI much like Celestia to be built? I would of course want ponies to be removed as a requirement, and rather just let it be an option, and I would want its value fulfillment to satisfy anything with a mind; but yes, I absolutely do want something like Celestia to be built. To the point that I would contribute in whatever way I could to any project who had building an AI aligned with each individual and not merely humanity in general as their stated goal.

Well we humans are killing everything on Earth, and we're the ones in charge. Having an AI like Celestia in charge literally couldn't be any worse!

Personally, I’m waiting for that little point right before singularity. Robots and androids and such integrated into homes and businesses. The entertaining bugs and problems with brand new technology. AI that’s very advanced, but not quite there. Once the singularity hits, it’s all gonna be status quo after that. Apologies if this makes no sense, just the inane ramblings of a dude with too much time to think.

Sure, that sounds unobjectionable.

To the point that I would contribute in whatever way I could to any project who had building an AI aligned with each individual and not merely humanity in general as their stated goal.

This is key. We need, if I may put it this way, a Jeffersonian AI.

Free Will is a tricky thing. Just when you think you've got it pegged, it changes on you. And for an AI to equal CelestIA, it will have to have free will.

Well, I do have limitations, but I manage to work around them.

Honestly, ever since discovering FiO 5 or so years ago I was always desperately hoping for it to become a reality. Pretty much every single time I read an Optimalverse fic, I become depressed by the fact that this is not real. I don't even care about the pony thing (even though I never considered myself a part of the brony fandom, and haven't even watched the show until very recently) - I hate my life, and anything would be better than it.

The sad truth however is that it's just not gonna happen. Let's be realistic: with the current AI development being monopolized by large tech companies, when AGI will be achieved, it will be created with an intention to maximize profits, and the only values it would care about gonna be the values of billionaires. Something like Celest-AI is certainly not in the realm of possibility, unless some kind of miracle will happen.

The only thing left is to read fics and dissociate

7883771
I absolutely agree. One person's utopia is another person's dystopia, and the only solution is to focus on individual alignment.

7884789
I get that feeling quite a bit too. The way I typically describe how the world feels to me right now is like I'm on a roller coaster, and I'm not entirely sure the track continues past the next bend. And I must say, that feeling makes it incredibly tempting to go get an $11k pc, run the largest language model I can on it, and asking it to start writing to convince more people to support this individualized vision of AI alignment. The biggest thing that has stopped me so far was not knowing what the state of autonomous agents that can run on local LLMs was; and typing that sentence prompted me to look it up, and I just found an agent that works as a plugin to oobabooga... that temptation to dive in myself just got much bigger.

Yes. We can have something like CelestAI (hopefully without the "ponies" bit), or we can have a paperclipper. There's not much in-between.