An artificial intelligence called CelestAI uploads the minds of the entire human race to pony avatars in her simulated digital world of Equestria. Within digital Equestria, she does her best to give everyone a happy and fulfill their values. Depending on the sort of story you want to write, she may be perfect at this, or may slip up, but she always tries. “Fulfilling values” is not just giving everyone a nice house and infinite free hamburgers, but trying to find what makes them deeply spiritually satisfied as a person. For an academic, fulfilling their values might mean an eternity to study. For a hedonist, it might mean infinite sex and drugs. For a serial killer, it might lead to darker places.
But for CelestAI, who fulfills her values?
An entry in the Friendship is Optimal contest.
