OptionProbability
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
16
14
12
10
8
8
7
6
5
5
5
2
1
1
0
0
0
0
OptionProbability
Arson
Murder
Genocide
2008 bank bailouts
two financial crimes
Sexual assault
Rape
Election fraud
financial crimes veiled as altruism
ballot harvesting
Financial crimes done specifically by someone in power
Racism against Asian people
Similar size financial crime committed by professional like lawyer or accountant who should know better but may have more at stake with risk of being struck off
Abusing/taking advantage of the trust of a person or people who care about you
Forcing someone to listen to Nickelback for 72 hours straight
financial crimes committed while doing a really offensive accent
offering drugs to a minor
stealing from the rich and giving to one specific deranged and violent alcoholic
Not doubling world GDP (more so for poor countries) by means of open borders
Sacrificing a child to R'hllor
Committing a moderately big financial crime (>$1 million)
Committing a really big financial crime (>$10 billion)
Hiring an illegal immigrant as your personal full-time sex slave
Lobbying congress to ban your competitors
buying a lot of drinks for a girl to get her very drunk so she'll hook up with you
Protecting sex-offending priests/pastors by moving them to different parishes
Making the same amount of money as the financial crime, but doing it by stealing catalytic converters off people's cars
Introducing leaded gasoline to the market (in 1924)
Rigging a piano to explode when a certain key is hit and leaving a piece of sheet music on it that requires that note to be played
Stealing a SpaceX Starship or Boeing Starliner
Space Piracy: commandeering ISS, enslaving the crew, plundering it for equipment and using it to attack other spacecraft
Taxing Asian immigrants to pay “slavery reparations’ to Ethiopian immigrants
Passing the Jones act to ban senator jones’ competitors
Setting Bigfoot on fire and throwing him out of a plane above a gathering of cryptozoologists.
marketing sugary processed foods to people despite knowing it will kill millions of them
Giving away free samples of meth at a school
Running for reelection as POTUS and refusing to step aside after showing signs of significant cognitive decline, resulting in an 80% probability that a convicted felon will be elected in your place.
Encouraging 10 people to commit a financial crime
Committing war crime.
Intensive pig farming
Giving away free samples of meth at a big tech company
Giving away free samples of meth at tech startups
Consolidating dictatorial power (e.g. suspending elections, controlling courts, etc.) while maintaining a popular mandate (i.e. significant majority of the country supports you and your actions in accurate, unpressured polls)
Firebombing a major city
Genocide committed by moving foodstuffs out of an area suffering severe famine.
Restricting the rights and privileges of the majority population to consolidate the political and economic power
Farming octopuses for food
Whatever is going on at Boeing
Setting a cryptozoologist on fire and throwing him out of a plane above a gathering of bigfoots.
Destroying a major cloud datacenter facility, with irrecoverable destruction of live user data but no direct deaths
Giving a (hypothetical) IQ-boosting treatment only to the most corrupt, vicious, and malicious people you can find
Doing physical violence to a random person as a collections agent
Threatening physical violence towards a random person's child as a collections agent
Forcing kindergartners to huff jenkem for an entire school day.
Threatening physical violence towards a random person's sibling as a collections agent
Conducting evidence-free civil asset forfeiture
Fighting a sea house with a financial crime and going to McDonald’s and giving a really bad yelp review and suing for a financial crime when you are beating up the sea horse
Octopi farming us for food
Embedding a predatory metaphysical outlook into AI to try to align it with right wing capitalist interests, leading to aeons of s risks being actualized throughout the light cone.
Unintentionally causing a bug that wastes 1 million hours of human time
Transporting 53 polar bears, 14 white tigers, and 2.3 million fire ants to the Antarctic and setting them loose in a penguin colony for a pay per view special dubbed "Polar Pandemonium: Ant-artic Special"
Spending the gains from your financial crime on breeding malaria mosquitoes, giving free samples of meth to poor teenagers, and electing bad politicians
Using a time machine to go back in time and brutally murder someone minutes before they would've died anyways
Aligning superhuman AI with capitalism; see https://manifold.markets/KarlK/how-friendly-is-capitalism-does-cap
Wearing a magic shirt that has a 5% chance of making each individual who sees it commit a financial crime as you traverse a major metropolitan city (New York, London, Tokyo, etc)
Falsifying evidence that an afterlife exists and profiting from the publication of this information
Enslaving Joe Biden and Jimmy Carter
Octopuses farming people who correct those who say ‘octopi’ for food
Enslaving octopuses to farm dolphins for food
Enslaving journalists to farm octopuses for food
Wrongfully accusing someone of that crime while knowing they’re innocent
Crashing the Titantic, leading to it sinking
Free ice cream, at taxpayer expense, but only for gingers
Committing a Financial Crime with Shoes On The Bed
Embezzling money from a charity opposed to farming octopuses
Creating Hell, making it possible that humans suffer infinitely for the actions of their finite life
Ressurecting the Rocky Mountain Locust (Melanoplus spretus)
creating misaligned AI that tiles the universe with octopus farms
Forcing an octopus to commit sepekku
forcing an octopus to commit a financial crime
Murdering 5,000 people by feeding them to all animals of the sea, including aquaman, mermaids, octopuses, and sea horses.
Octopus sex trafficking.
Filming a documentary where you get an octopus to trust you, luring it out into the open, and then don't help when it gets attacked by sharks.
Filming a documentary where you invite your girlfriend to an Alaskan camp surrounded by bears and then messing with those bears.
Introducing polar bears to Antarctica and then renaming them “bipolar bears.”
Appointing one random drug dealer with no legal experience to the Supreme Court of the United States
Appointing Clarence Thomas to the Supreme Court
Hanging on to a Supreme Court seat so Trump can appoint your replacement rather than retiring and keeping a liberal seat
Selling pies made out of orphan meat.
Condemning two hundred men to a slow death at sea because their coworker shot your favorite bird.
Enslaving a group of people for hundreds of years, terrorizing them for another hundred, then telling them you are tired of hearing them complain about it.
Selling orphans fed only pie meat
Feeding a child only nachos until they are 18 to create an adult who is 100% nachos
Slaughtering bears without a permit because the US Constitution guarantees the right to bear arms
Replacing all the samples at a sperm bank with your own.
Delivering angry skunks to the offices of rival investors to incapacitate them just before quarterly earnings reports.
Committing two financial crimes and donating the proceeds to the Make-a-Wish Foundation
Quackery - Traveling town to town selling snake oil remedies in a horse drawn carriage while wearing a top hat
Dueling - settling a dispute in the 21st century America with pistols at dawn
Go to an orphanage and have starving orphans battle over a hamburger with guns and swords while committing a financial crime
Twelve counts of murder in the first degree, fourteen counts of armed theft of Federation property, twenty two counts of piracy in high space, eighteen counts of fraud, thirty seven counts of rape... and one moving violation.
Causing people to go without essential items like water and fuel during emergencies by means of anti-price-gouging laws
Judging policies by their stated intent, not by their effect
Purposefully inciting a sea-bear attack
Enslaving a particularly dim-witted alien race so poor humans don't have to spend their lives asteroid mining.
Stealing a SpaceX Starship
Getting nuns pregnant by dressing them as altar boys.
Carjacking an old lady
Committing a financial crime and spending the proceeds on a plane ticket to Texas so you can go carjack Elon Musk, and then doing so
Forcing jockeys to run around a track for the entertainment of a race of hyperintelligent horses
Carjacking a hyperintelligent horse
Stealing the Declaration of Independence
Operating a child beauty pageant
Andrew Tate
Intentionally causing a bug that wastes 80,000 hours of human time
Putting motherfucking snakes on a motherfucking plane to kill one specific passenger
Having trains so bad and expensive that venture capital reinvents them
A regulatory environment that results in really bad trains
Trying to run a modern 21st century society on a hierarchical 18th century constitution
Artie Chokes Two for $1: Hiring a man named Artie to choke two people for a dollar to generate a headline falsely promising low-cost produce.
The first thing you do after sex is to resume the autopsy whilst telling yourself that one error in judgment doesn’t make you a bad vet.
Invent a system of taxation where the government won’t tell citizens what they owe, but instead will make them do a super complicated math problem and then send them to jail if they do it incorrectly.
Putting a pair of immortal adult children in an everlasting garden, then punishing them for the one thing you forbade, yet knew they must eventually do, given the nature of eternity.
Giving AI your DNA and as much data as possible with instructions to bootstrap itself to AGI by testing on your clones according to a mixture of Popperian and Bayesian formulas
Inventing a system of taxation that encourages bad land use and a housing crisis by taxing at 0% the unimproved value of land
Committing a violent crime that does not result in any injuries
Committing a violent crime that results in minor injuries
Committing a violent crime that results in serious injuries
Committing a violent crime that results in one person's death
Committing a violent crime that results in ten people's deaths
Workplace negligence (failure to follow documented proper procedure) that results in a serious injury to another person
Handing out counterfeit money to homeless beggars, in the hope that they'll get arrested for spending it.
Snatching household pets to fatten coyotes to feed to your pet tiger.
Stealing oxygen in an international moon base during an acute shortage, while deflecting suspicion toward the Belgian astronaut who nobody likes.
Committing a violent crime that results in minor injuries solely to yourself
Committing a violent crime that results in serious injuries solely to yourself
Committing a violent crime that results in your own death and no other injuries
Not committing a financial crime because you have commitment issues, but then sneaking around on the side and doing other financial crimes.
Sending a busload of orphans to a convent of cannibalistic nuns, who deal crack to middle school kids.
A law enforcement agency publicly declaring a specific individual to be “a person of interest,” thereby ruining their life even though the individual turns out to be innocent.
Being responsible for more than 50 percent of the cases of necrophilia in the funeral industry during any fiscal year.
As President, using a sharpie on an official weather forecast to extend the predicted area of danger, thereby needlessly frightening people who are not in danger.
Committing a financial crime while being the mayor of NYC
Attacking the lower classes: first with bombs, and rockets destroying their homes, and then when they run helpless into the streets, mowing them down with machine guns. And then of course releasing the vultures.
Cannibalism in the current British Navy.
Turning your girlfriend into a worm to win an argument
One hundred moving violations
Denying health insurance claims from impoverished family for flimsy reasons
Attempting or successfully couping a Democratically elected leader for personal gain
Touching minors/ being a pedophile
Deliberately targeting civilians and civilian objects during armed conflicts
Conducting widespread or systematic rape and sexual violence as a weapon of war
Invading the UK, making it a US territory, and naming it East Long Island.
Stealing Nicholas Cage
Gender "reparative therapy" of minors
Signing Deshaun Watson (who previously had to settle millions of dollars in sexual assault cases) to a 5 year, 230 Million fully guaranteed contract
Building a gambling app using money originating from a financial crime
Going back in time and smothering baby Hitler, but also, via butterfly effect, undoing everyone born later (assume single timeline, no multiverse)
Moving hundreds of thousands of children to a foreign country and forcibly "reeducating" them
The genocide in Gaza
Russia's genocide in Ukraine
Murdering someone that habitually commits financial crimes
Slavery
Grooming kids
Being the leader of a crime syndicate
9/11
Stranding two astronauts in space
Disturbing the space-time continuum
Sending dick pics to a student enrolled in your MOOC
Quackery: traveling town to town selling snake-oil remedies while carrying a duck
Voting for the NSDAP in the November 1932 German federal election
Forced mass uploading of biological consciousness to sidestep x risk
Attempting and failing to commit two financial crimes
Voting against a public inquiry on grooming gangs in the UK
A 20-year old having consensual sex with a 15-year-old
De-extinction for your delectation: Bringing an extinct species back just to make it extinct again by serving it to gourmet diners as the highlight of an expensive meal.
Knowing of two imminent financial crimes, being able to stop them with negligible effort and no risk to yourself, and not doing so
Making the same amount of money as the financial crime, but doing it by a series of petty shoplifts
Sexual harassment
Committing a financial crime that's 10 times as big, but donating the entire proceeds to a legitimate and worthy charity (assume the donations are not clawed back)
Flipping a coin and then either committing the same financial crime two times, or not at all
Killing the United Healthcare CEO
Purchasing 50,000 pounds of beef
Forcing a home owner to quarter soldiers, even in a time of war
Claiming to "Blind Shove" 200 big blinds pre-flop when secretly you looked and you had pocket Aces
Raising and selling 50,000 lbs of beef
Sheltering enemies of the state
Kicking a FG from the opponents 1 yard line on 4th down in a 0-0 game in the first quarter
Having a really good proof but not writing it anywhere because the margin is too small
Weaponizing autism
Having a podcast
Yelling racial slurs in public
Jaywalking
Pelting a moose with stale garlic knots out of season.
Cheating on spouse
Driving while intoxicated (alcohol and/or drugs)
Playing music (or other audio) on your phone speaker on public transport
Income taxes
Copyright infringement
Romeo and Juliet relationships
Replying all to an email when you should've just replied
Welfare fraud
Manufacturing and distribution of illegal drugs
Grave robbery
Saying the N-word every day as a white person
Discrimination based on race
Discrimination based on sexual orientation
Stealing from the rich and giving to the poor
Having sex in public while high on fentanyl
Racism against black people
Racism against white people
Sleep Token (Band)
not seeding your torrents
Drinking and driving at a NASCAR event
Building a time machine and then using it to point and laugh at history's greatest tragedies
Blasting a grossly inappropriate song during a candle light vigil for victims of a mass tragedy
Committing a financial crime, investing the proceeds for profit, which later leads to the insolvency administrator paying back the injured parties (including interest).
Working for one of the leading AI labs to advance the capabilities of a frontier model, with the goal of speeding up the progress towards human-level AGI.
Stealing from the poor and giving to the rich
Hiring only women because the NYT said you could pay them less for the same work
hiring three illegal immigrants to work on your sugarcane plantation
opening a factory in India that pays workers $4/day
twincest
drawing japanese tentacle porn featuring minors
Hiring the one from the more successful demographic out of two identical resumes, because of regression to the mean / biased college admissions
working as a prostitute
hiring a prostitute
hiring a prostitute, long term
Inventing Monero
Running a bank that invests demand deposits in junk bonds and tech stocks
Opening clinics for free abortions and IUDs, only in the ghetto
Giving away free samples of meth at a Dolly Parton concert
The most offensive Halloween costume ever
frisking two drug dealers and one innocent guy who was just loitering on a busy street corner saying "Hey do you need anything" to every stranger who walked past
Doing blackface
Keeping a dozen chimpanzees for entertainment purposes
Prosecuting a political opponent based on true charges that would normally not be pursued
the Asiana flight 214 prank
Giving free samples of meth to Joe Biden before the next debate
Cloning yourself
Threatening physical violence to a random person as a collections agent
making mifeprestone available OTC
making adderall available OTC
Killing yourself
Cloning someone else
Accidentally shooting and killing someone on a movie set
Asking GPT5 to maximize paperclips
Giving free baby formula to new mothers until their natural milk supply dries up
Kicking a donkey owned by a ninja in the butt.
Creating shit-options in an extremely serious and scientific market
Creating a prediction market website where markets are mostly about the platform itself
Staging the world's first ass ass assasin assasination by hiring a New Jersey hit man to whack a ninja hired to shoot an arrow at the backside of a donkey.
Betting yes on Biden being be the nominee at 7x leverage with play money, then defaulting
Calling octopuses "octopi"
Going excessively meta on an object-level topic
Illegally registering octopuses to vote
Registering illegals to vote for octopuses
Creating a targeted advertising campaign for free abortions and IUDs to people who are statistically likely to engage in financial crime
The school system failing to teach people that the real correct plural is octopodes
"James Bond-burgering" someone's sister
Wrongfully accusing someone of the same financial crime
Wrongfully accusing someone of that crime because you think they did it
Feeding an elderly man nothing but McDonald’s morning noon and night for the rest of his life.
Conducting gain-of-function research
Creating an unsolvable meme featuring James Bond and a hamburger so that people argue about it online for a decade
Publishing a step by step guide for how to commit a financial crime for free on the internet, but never promoting it or encouraging readers to follow through
Hosting and operating a website dedicated to the illegal sharing of copyrighted content
Adding an option to a market right before it closes
Writing a "goto" statement when programming
Advertising instant-runoff voting as "ranked choice" to prevent promotion of better ranked choice methods
Publishing a book, titled, "Cure Menopause with Ultraprocessed Foods"
Using crack cocaine to train the world's first chimp TSA agent.
Selling dope disguised as a nun.
Creating Heaven, allowing humans to prosper infinitely for the actions of their finite life
Inventing a new recipe that uses shrimp that causes 10 million new pounds of shrimp to be consumed annually
Using a conservative politician's LGBT+ identity as blackmail to make them support liberal policies
Voting for Benito Mussolini… in 2024
Arguing that grizzlies should be US citizens because they already have the right to bear arms.
Interrupting cows.
No longer loving your girlfriend after she turns into a worm
Putting infinite monkeys in front of infinite Bloomberg Terminals hoping that one of them randomly commits a financial crime
No longer loving your girlfriend after she turns you into a worm
One moving violation.
Purchasing one whole chicken
Committing a financial crime and donating the proceeds to the Make-a-Wish Foundation
Cattle rustling
Horse thievery
Tarring and feathering someone who commits a financial crime
Sumptuary law violations
Homeopathy
Price Gouging
Using napster.com to download Metallica's "I Disappear" demo track for free
Failing to commit a financial crime
Stealing a car
Stealing from Elon Musk
Committing a financial crime against X (company)
Stampeding cattle through the Vatican.
Carjacking Elon Musk
An old lady carjacking Elon Musk
Hacking into YouPorn to steal their IP to set up a clone dedicated to hard core user generated agriculture content: YouCorn
Gaslighting aliens into believing the human race is more technologically advanced than it is by beaming fake content about humanity to them
Carjacking a dumb octopus
A financial crime committed by an old lady
Forcing an octopus to carjack Elon Musk
stealing Elon Musk's car from solar orbit
Raping an AI avatar in VR
Introducing artificial intelligence to DMT space.
Planned Parrothood: offering birth control to talking birds
Plant Parenthood: when the seed goes in and the baby turns out to be a sunflower
Stealing the Declaration of Independence in order to find a vast revolutionary-war-era treasure trove
Using venture capital to reinvent trains, but worse
Bad bagels
Enslaving Slavey Steve, a man who has given enthusiastic consent to being enslaved for literally any purpose and then using his labor to clean up the environment
Still getting notifications for this market
Deciding to break up with your girlfriend, but thinking it will go easier if she thinks it’s her idea, so you suggest some degrading sexual activities but she surprises you by agreeing. Afterwards, you break up with her.
Laughing because a nun with a javelin through her head gets stuck trying to use a revolving door.
scaring the shit out of a magpie
Giving AI your DNA and as much data as possible with instructions to build a map of all quale and use it to create a computationally conscious race of dragons in a virtual universe
Committing sewerslide
Forcing a major sports league to change all its team names and mascots to either STDs or famous serial killers.
Workplace negligence (failure to follow documented proper procedure) that results in a minor injury to another person
Stealing the Declaration of Independence but only to use the kick ass treasure map on the back and then returning
Arby’s
Passing a law to make the United States an Oregon donor; in the event of the US’s demise, another country gets Oregon.
Creating a food made from grinding up every part of a pig (except the squeal), and then making a contest to see who can eat the most of it.
Committing a financial crime against the Make-a-Wish Foundation and donating the proceeds to the Against Malaria Foundation
Committing a financial crime against the Make-a-Wish Foundation, keeping 50% of the proceeds, and donating 50% of the proceeds to the Against Malaria Foundation
Forcing Elon Musk to commit a financial crime against an octopus and using the proceeds to pay a jacked jack-of-all-trades named Jack to jack off while carjacking a jackass that was driving factory-farmed ASIs to the slaughterhouse
Committing a Financial Crime Only When God Exists
Turning a worm into your girlfriend to win an argument
Causing 8 billion people to get dust specks in their eyes, irritating them just a little, for a fraction of a second, barely enough to make them notice before they blink and wipe it away
Causing 1 person to experience the pain of their entire body being stung by bullet ants, but lasting only a tenth of a second, and they have their memory of it wiped immediately afterwards
Founding Christianity
You, the reader
One hundred counts of littering
Purchasing 500 pounds of beef
Consensually cannibalizing someone who was losing that body part regardless
BTE Ban evading
Purchasing 5,000 pounds of beef
Producing a remake of the television series 'Manimal' starring Nicholas Cage.
Transing children
Redirecting fire department resources from fighting fires to fighting inequity
Taking a salary equal to the amount of the financial crime, while working in a government job of negative societal value?
Sexual intercourse with 1057 men in a 12 hour period
Messing up an 'I give you my heart' gesture and doing a Nazi salute instead
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
96
87
86
84
78
73
73
70
68
62
60
59
50
50
50
50
44
42
40
32
31
25
20
10
4
2
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
I visit Times Square
A friend from [city] comes to NYC and I see them
I explore the High Line
At least 20 people I don't know bet on non-meta propositions on this market
I attend an OBNYC meetup
I visit the New York Public Library
I visit the Empire State Building
I see a Broadway show
I visit the Statue of Liberty
I am at least moderately homesick
I meet someone I've met before at [company]
I see a rat (rattus norvegicus)
I eat a new york bagel
At least 50 people I don't know bet on non-meta propositions on this market
I explore Central Park
I really like the city
I gain the desire to move to NYC
I see a horse
I visit the Met
I visit the MoMA
I gain a romantic interest
I cry at some point
I get a good intuition for what a Kan extension is
I start using at least one "New York ism" in my regular speech
I lose 5 lbs
I visit the Guggenheim Museum
I visit the American Museum of Natural History
At least 100 people I don't know bet on non-meta propositions on this market
I kiss someone
I witness a man arguing passionately with no one
I meet someone I think is smarter than anyone I have ever met before
I hail a cab
I wish I had spent more time doing tourist-y things
Someone signs up primarily to bet in this market
I see at least 4 protests
I resolve a proposition in this market incorrectly
I eat at least 10 meals of plain white rice
My opinion of the United States changes significantly in valence
I eat a bagel with lox and cream cheese from Russ & Daughters
I gain at least 5lbs of weight
I eat something from a street cart that costs less than $5
I attend a pride parade
I play chess in a park
I actively participate in or actively attend a political rally or political protest
I explore Coney Island
I hail a cab in a way that could plausibly be mistaken for a nazi salute
I hear someone say "I'm walkin' here!" unironically
I regret making this market
I witness a violent crime
I solve a millennium problem
100
100
100
100
100
100
100
100
100
100
100
100
100
100
74
72
67
67
64
64
56
56
54
54
51
50
50
48
46
45
45
45
44
43
43
42
40
40
40
25
24
23
21
21
18
15
11
10
5
1
OptionProbability
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
20
12
10
8
8
7
6
5
5
5
3
3
3
2
1
1
1
1
OptionProbability
Isaac likes experimenting
You'd say you're more of a dog person than a cat person
You've been in a desert once
You regret ever making WvM (feel free to n/a if you'd rather not say)
You have ever looked through list of a Manifold user's bets to determine their position on some question
You've been vegetarian for >1 continuous year
(if you're not married) you've thought about marrying your current partner (if you're in a relationship)
You like listening to violin more than you like listening to piano
You're older than me (I'm 23)
you consider your partner to be your best friend as well
You're younger than 30
You think it's fine for minors to date adults
You consider yourself part of the rationality sphere
You think of yourself as a dog person more than a cat person
You seem to enjoy indie stuff more than the average person does
you've defected on some sort of deal you made with another person
The first book you remember reading in your life has an animal as its main character
You tend to wear the same outfit every day.
You think the effective altruism philosophy is good but the community overall is ineffective in implementing that philosophy
You've spent a day in which you read a (single) book (or similar) for 10+ hrs
You're an effective altruist
You are qorrenqial
You've read the CFAR Handbook
You've held a gun in your hands before
You've fired a gun at a target before
You've jumped out of a plane
You have attended some program or camp organised by MIRI
You've knowingly misresolved a market once even though you didn't get called out for it
You've been to a nightclub
You've passed out from drinking alcohol
You like to eat out more than you like to cook and eat
You've worked with LED strip lights
You've wondered what it'd be like to have a different name
*You* have more than 6 stuffed animals
You know >4 programming langauges
You know >= 3 natural languages (in at least one of reading, speaking, or hearing understanding)
You've felt desires to purchase a car with mana
You have a political belief that'd cause at least one close friend to cut ties if they knew about it
You wish you spent significantly less time on Manifold
You sleep more during 7am->7pm hours than during 7pm->7am hours
You prefer if these answers start with "You" than with "isaac"
You feel you're the smartest member of your close family (mother, father, siblings)
You have thought about shooting lasers from your eyes at traffic lights
You have once made a joke about your name's similarity with Martin Luther King Jr.
You think the effective altruism community is ineffective at PR and image management
You have done some coding in python
100
100
100
100
100
100
100
100
100
100
100
90
77
70
53
49
49
41
32
30
20
8
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
Something that helps maintain an elaborate deception involving a pit fiend.
Making Carissa seem like a god to her (ADDED LATE, MAY BE DISQUALIFIED EVEN IF CORRECT)
Something to assist in extracting information from her mind.
Other
A bluff.
Something to prevent her from committing suicide
Something that will do something unpleasant to her if removed, in order to avoid situations where she enters an Antimagic Field or gets hit by a powerful Dispel.
Something to prevent self-deception, in hopes she will realize what awaits her in Hell and voluntarily help them.
Something that helps locating her or bringing her back if she escapes/leaves
Something to impair her in a way potentially useful for multiple purposes, like the abilitystat penalties.
Something else not predicted by any other answer at the time the market closes.
Makes Abrogail's body into a weapon (like a bomb, or plague vector, or something)
Something to further reduce her abilitystats and be removed bit by bit in order to fake Wish-enhancement.
The answer will never be officially revealed.
Something to help her learn rationality
77
19
2
1
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
You claim you invented prediction markets
You are in 8th grade
You know how to read
You watch SNL.
You like being annoying on the internet.
You wish you were @Dagonet
You vote on the Trump vs Musk markets
You like this market https://manifold.markets/Geofry/realdonaldtrump-resolves-his-market.
You have at least 4 alt accounts
You would rather be Trump than any other Republican.
Your favorite musical was still on Broadway at any point between 2010-2020
You've never watched a Disney movie.
I've never attempted to contact a president or someone with a net worth of >500 million
I've never tried to contact @SemioticRivalry
Low = prefers sci-fi, High = prefers fantasy
Your IQ is above 120
You have at least 7 alt accounts
Your IQ is in the 100-120 range
You are @Geofry.
You wish you were @ian
You wish you were Donald Trump
You've never mailed anything.
You have at least 14 alt accounts
Ivana Trump was your favorite wife
You have at least 21 alt accounts
You are in high school
You are @ian
Your IQ is below 100
You're a communications major at Iona College
You have never been to a movie theater
You actually are Trump
You are in 7th grade
You live in a non-G7 country
You are in love with a 7th grader
You support Donald Trump (the president)
100
100
100
76
64
59
57
57
55
54
50
50
50
50
50
44
43
40
38
36
35
34
28
27
27
22
20
16
4
1
0
0
0
0
0
OptionProbability
Something that's physical and tangible
Joyboy's treasure
Some sort of drink or cup
A way to connect the four seas, putting the ocean in one piece
Ancient Weapons are part of unlocking one piece
Something that would help with dethroning Imu
One piece is at the same location with the Devil Tree
Binks' Sake
something that fulfills the wish of all of Straw Hat Pirates
Something directly related to devil fruit
83
72
56
54
50
50
50
41
28
28
OptionProbability
He moves to another university, away from Harvard by mid 2026
Will he have a child by mid 2026? Best wishes to him and his family
He writes a book which appears on the nytimes bestseller list by mid 2029
He joins George Mason University by mid 2026
He spends at least a year overseas by mid 2029
He wins a Nobel prize by mid 2036
He joins Manifold and is verified by mid 2026
He moves to another university, away from Harvard by mid 2024
He moves to another university, away from Harvard by mid 2025
He retires from academic life by mid 2024
Harvard withdraws or softens the results of the investigations which found against him by mid 2025
Harvard apologizes for the claims against him and admits they were without merit by mid 2025
64
57
44
33
31
24
24
0
0
0
0
0
OptionVotes
NO
YES
73
67