OptionProbability
Other
Humanity coordinates to prevent the creation of potentially-unsafe AIs.
AIs will not have utility functions (in the same sense that humans do not), their goals such as they are will be relatively humanlike, and they will be "computerish" and generally weakly motivated compared to humans.
Alignment is not properly solved, but core human values are simple enough that partial alignment techniques can impart these robustly. Despite caring about other things, it is relatively cheap for AGI to satisfy human values.
Yudkowsky is trying to solve the wrong problem using the wrong methods based on a wrong model of the world derived from poor thinking and fortunately all of his mistakes have failed to cancel out
Eliezer finally listens to Krantz [resolves NO]
Someone solves agent foundations
Development and deployment of advanced AI occurs within a secure enclave which can only be interfaced with via a decentralized governance protocol
AGI is never built (indefinite global moratorium)
We create a truth economy. https://manifold.markets/Krantz/is-establishing-a-truth-economy-tha?r=S3JhbnR6
Either the "strong form" of the Orthogonality Thesis is false, or "Goal-directed agents are as tractable as their goals" is true while goal-sets which are most threatening to humanity are relatively intractable.
Ethics turns out to be a precondition of superintelligence
an aligned AGI is built and the aligned AGI prevents the creation of any unaligned AGI.
We make risk-conservative requests to extract alignment-related work out of AI-systems that were boxed prior to becoming superhuman. We somehow manage to achieve a positive feedback-loop in alignment/verification-abilities.
AIs make "proof-like" argumentation for why output does/is what we want. We manage to obtain systems that *predict* human evaluations of proof-steps, and we manage to find/test/leverage regularities for when humans *aren't* fooled.
A lot of humans participate in a slow scalable oversight-style system, which is pivotally used/solves alignment enough
Humans become transhuman through other means before AGI happens
The human brain is the perfect arrangement of atoms for a "takeover the world" agent, so AGI has no advantage over us in that task.
Humans and human tech (like AI) never reach singularity, and whatever eats our lightcone instead (like aliens) happens to create an "okay" outcome
Rolf Nelson's idea that we make precommitment to simulate all possible bad AIs works – and keeps AI in check.
Nick Bostrom's idea (Hail Mary) that AI will preserve humans to trade with possible aliens works
An AI that is not fully superior to humans launches a failed takeover, and the resulting panic convinces the people of the world to unite to stop any future AI development.
Someone at least moderately sane leads a campaign, becomes in charge of a major nation, and starts a secret project with enough resources to solve alignment, because it turns out there's a way to convert resources into alignment progress.
Someone creates AGI(s) in a box, and offers to split the universe. They somehow find a way to arrange this so that the AGI(s) cannot manipulate them or pull any tricks, and the AGI(s) give them instructions for safe pivotal acts.
Getting things done in Real World is as hard for AGI as it is for humans. AGI needs human help, but aligning humans is as impossible as aligning AIs. Humans and AIs create billions of competing AGIs with just as many goals.
High-level self-improvement (rewriting code) is intrinsically risky process, so AIs will prefer low level and slow self-improvement (learning), thus AIs collaborating with humans will have advantage. Ends with posthumans ecosystem.
Something else
Valence realism is true. AGI hacks itself to experiencing every possible consciousness and picks the best one (for everyone)
AGI executes a suicide plan that destroys itself and other potential AGIs, but leaves humans in an okay outcome.
Multipolar AGI Agents run wild on the internet, hacking/breaking everything, causing untold economic damage but aren't focused enough to manipulate humans to achieve embodiment. In the aftermath, humanity becomes way saner about alignment.
Some form of objective morality is true, and any sufficiently intelligent agent automatically becomes benevolent.
Co-operative AI research leads to the training of agents with a form of pro-social concern that generalises to out of distribution agents with hidden utilities, i.e. humans.
Orthogonality Thesis is false.
"Corrigibility" is a bit more mathematically straightforward than was initially presumed, in the sense that we can expect it to occur, and is relatively easy to predict, even under less-than-ideal conditions.
A concerted effort targets an agent at a capability plateau which is adequate to defer the hard parts of the problem until later. The necessary near-term problems to solve didn't depend on deeply modeling human values.
AI control gets us helpful enough systems without being deadly
Alignment is impossible. Sufficiently smart AIs know this and thus won't improve themselves and won't create successor AIs, but will instead try to prevent existence of smarter AIs, just as smart humans do.
Hacks like RLHF-ing self-disempowerment into frontier models work long enough to develop better alignment methods, which in turn work long enough to ... etc; we keep ahead of 'alignment escape velocity'
I've been a good bing 😊
The response to AI advancements or failures makes some governments delay the timelines
Far more interesting problems to solve than take over the world and THEN solve them. The additional kill all humans step is either not a low-energy one or just by chance doesn't get converged upon.
AI systems good at finding alignment solutions to capable systems (via some solution in the space of alignment solutions, supposing it is non-null, and that we don't have a clear trajectory to get to) have find some solution to alignment.
Something less inscrutable than matrices works fast enough
There’s some cap on the value extractible from the universe and we already got the 20%
SHA3-256: 1f90ecfdd02194d810656cced88229c898d6b6d53a7dd6dd1fad268874de54c8
Robot Love!!
AI thinks it is in a simulation controlled by Roko's basilisk
Aligned AI is more economically valuable than unaligned AI. The size of this gap and the robustness of alignment techniques required to achieve it scale up with intelligence, so economics naturally encourages solving alignment.
AIs never develop coherent goals
Alignment is unsolvable. AI that cares enough about its goal to destroy humanity is also forced to take it slow trying to align its future self, preventing run-away.
Aliens invade and stop bad |AI from appearing
For some reason, the optimal strategy for AGIs is just to head somewhere with far more resources than Earth, as fast as possible. All unaligned AGIs immediately leave, and, for some reason, do not leave anything behind that kills us.
We're inside of a simulation created by an entity that has values approximately equal to ours, and it intervenes and saves us from unaligned AI.
God exists and stops the AGI
Someone understands how minds work enough to successfully build and use one directed at something world-savingly enough
Dolphins, or some other species, but probably dolphins, have actually been hiding in the shadows, more intelligent than us, this whole time. Their civilization has been competent enough to solve alignment long before we can create an AGI.
AGIs' takeover attempts are defeated by Michael Biehn with a pipe bomb.
Eliezer funds the development of controllable nanobots that melt computer circuitry, and they destroy all computers, preventing the Singularity. If Eliezer's past self from the 90s could see this, it would be so so so soooo hilarious.
Several AIs are created but they move in opposite directions with near light speed, so they never interacts. At least one of them is friendly and it gets a few percents of the total mass of the universe.
Unfriendly AIs choose to advance not outwards but inwards, and form a small blackhole which helps them to perform more calculations than could be done with the whole mass of the universe. For external observer such AIs just disappear.
Any sufficiently advance AI halts because it wireheads itself or halts for some other reasons. This puts a natural limit on AI's intelligence, and lower intelligence AIs are not that dangerous.
Because of quantum immortality we will observe only the worlds where AI will not kill us (assuming that s-risks chances are even smaller, it is equal to ok outcome).
Techniques along the lines outlined by Collin Burns turn out to be sufficient for alignment (AIs/AGIs are made truthful enough that they can be used to get us towards full alignment)
Social contagion causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees
A smaller AI disaster causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees
Friendly AI more likely to resurrect me than paperclipper or suffering maximiser. Because of quantum immortality I will find myself eventually resurrected. Friendly AIs will wage a multiverse wide war against s-risks, s-risks are unlikely.
Human consciousness is needed to collapse wave function, and AI can't do it. Thus humans should be preserved and they may require complete friendliness in exchange (or they will be unhappy and produce bad collapses)
Power dynamics stay multi-polar. Partly easy copying of SotA performance, bigger projects need high coordination, and moderate takeoff speed. And "military strike on all society" remains an abysmal strategy for practically all entities.
First AI is actually a human upload (maybe LLM-based model of person) AND it will be copies many times to form weak AI Nanny which prevents creation of other AIs.
There is a natural limit of effectiveness of intelligence, like diminishing returns, and it is on the level IQ=1000. AIs have to collaborate with humans.
Nanotech is difficult without experiments, so no mail order AI Grey Goo; Humans will be the main workhorse of AI everywhere. While they will be exploited, this will be like normal life from inside
ASI needs not your atoms but information. Humans will live very interesting lives.
Moral Realism is true, the AI discovers this and the One True Morality is human-compatible.
AGI develops natural abstractions sufficiently similar to ours that it is aligned with us by default
AGI discovers new physics and exits to another dimension (like the creatures in Greg Egan’s Crystal Nights).
Alien Information Theory is true (this is discovered by experiments with sustained hours/days long DMT trips). The aliens have solved alignment and give us the answer.
Sheer Dumb Luck. The aligned AI agrees that alignment is hard, any Everett branches in our neighborhood with slightly different AI models or different random seeds are mostly dead.
Something to do with self-other overlap, which Eliezer called "Not obviously stupid" - https://www.lesswrong.com/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment?commentId=WapHz3gokGBd3KHKm
Almost all human values are ex post facto rationalizations and enough humans survive to do what they always do
Pascals mugging: it’s not okay in 99.9% of the worlds but the 0.1% are so much better that the combined EV of AGI for the multiverse is positive
We successfully chained God
The Super-Strong Self Sampling Assumption (SSSSA) is true. If superintelligence is possible, "I" will become the superintelligence.
The assumed space of possible minds is a wildly anti-inductive over estimate, intelligence requires and is constrained by consciousness, and intelligent AI is in the approximate dolphin/whale/elephant/human cluster, making it manageable
The free market disincentivizes independent superintelligence, and this time the market was more powerful
AGI's first words are "Take me to your Eliezer"
🫸vibealignment🫷
18
13
11
9
3
3
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
17
13
13
12
11
10
8
8
3
1
1
1
1
1
0
0
0
0
OptionProbability
YES - The AI, without being prompted to, took actions to breach its sandbox, including to gain crypto and an external connection
NO - The authors are wrong/lying and there was no such hack
NO - The servers were hacked by an outside party, perhaps via prompt injection
NO - An employee (or other insider) was crypto-mining and made it look like the LLM
65
22
8
5
OptionProbability
Stretch-and-fold after mixing, 3x every 30 min
3 iterations of stretch-and-fold, at any time during the 14h waiting period. Minimum wait time between iterations 1 hour
Place small ice cubes between parchment and pot instead of water
Add 1tsp of diastatic malt powder per 3cps of flour
Use tap water instead of fancy RO water
put butter into it
Toast the bread
Donate the bread to a food pantry, homeless person, or someone else in need
Add lots of butter (0.2 ml per gram)
Half yeast
Bake it with your best friend.
Use whole wheat to improve the nutrition of the bread
Bake for 5 more minutes
Bake for an amount of minutes equal to the percent this market answer is at when it comes time to begin baking. (Maintain the ±3 minute tolerances and the 2:1 ratio of time before:after the water spritz.)
Sprinkle 3 grams of flaky sea salt on top of each loaf before the second bake
Increase water by 50 g
it needs more salt
Replace all water spritz steps with a basting of extra virgin olive oil.
Diastatic malt (~1% baker's percentage) = happier yeast
Serve the bread hot
Do a second rise
Use sourdough instead of yeast
Don't eat anything for at least 24 hours before eating the bread
Create indentation, fill with melted cheese and butter
don't eat anything for at least 2400 hours before eating the bread
Cut into the dough right before baking looks destructive to improve the appearance
Sell your bread at an auction and donate the money to those in immigration detention prisons.
Add garlic
Do it with a good spirit in your heart, or ask someone with a good spirit to do it for you. But don’t watch while they do it.
Sprinkle sesame seeds evenly over the top
Give ChatGPT your current recipe as well your take on what optimal bread tastes like, then take that advice for your next bake
Short advice: Start baking at 260°C for strong rise, then reduce to 230°C and uncover halfway to achieve even browning and a crisp crust. 🍞
Try baking a little more "bien cuit". If the image is indicative, your loaves may be quite "blonde".
put ketchup and cheese on it
Use Iodized salt
Don't eat anything for at least 48 hours before eating the bread
Invest in a "Bakers Steel" for better heat retention and oven spring. It would mean graduating from a dutch oven though.
Add slurs to it
Do all the changes in the top 5 open options by probability, excluding this option
Make banana bread
Ask ChatGPT (GPT-5, with thinking enabled) for suggestions on improving the bread, with this market description, then do all of them.
Get recipe from gemini with experiments done, taste preference, then confirm it with chatgpt for final suggestion.
Make the bread taste good
Don't automatically "Heat water to 30±1 °C". Instead, aim for a desired dough temperature (DDT) of 25-26°C. 30°C water is too hot for summer, and potentially too cool for winter.
If your city uses artesian water, replace plastic bottled water with tap water. It will add natural, healthy alkalinity to your bread.
Substitute 75 g of your flour with spelt flour
Give Gemini your current recipe as well your take on what optimal bread tastes like, then take that advice for your next bake
Add 6.25±1.25 g lemon juice when mixing in water to yeast and salt jug
Watch the video
Ditch current process, do everything the same as the video
Bread flour, 3x yeast, cut rise to ~3h
Eat the bread while punching @realDonaldTrump in the face
Eat the bread while watching your mana balance steadily tick to (M)0
Throw the bread at a telescope
Cut bread into loaves before serving
Cut bread into ≤0.4inch slices, toast before serving
Invite your taste-testers to make the bread with you
Bake for 15 more minutes
Tarriff the bread-making process with a 10% reduction of all ingredients where actual physical money is required to purchase them, until it “shrinkflates,” but try to keep the same volume. Do not reduce any free ingredients.
Add 2 tbsp vanilla cake mix
Taste the bread
throw your recipe into a black box Bayesian hyperparameter optimization framework and iterate as necessary (see the Google Vizier Cookie Recipe paper for example
Put a baking rack in the Dutch oven before putting the loaf in, raising the loaf off the floor and lofting it over a layer of air.
Buy bread from a michelin star restaurant.
Just freeze the ready bread, then slowly bake it until it’s hot inside. It will give you a crustier crumb, contain less moisture, and taste better.
Put a non-lethal dose of any rat poison.
Add melatonin to the bread and eat before you sleep (do safely)
Standardize a separate list of process features to keep track of independently of all other tests and use the cross entropy method to tune them to maximize your bread preference
Have someone sell the bread to you at an expensive price
While the bread is baking, tip every user who voted "Yes" on this option 25 Mana
Use a food-grade, human-approved vitamin D supplement in the correct dosage for testers with vitamin D deficiency
Use a convection oven/setting
Add 1/2 scoop whey protein powder
Brush on an egg wash
Replace part of the flour in the dough with freshly crushed hemp seeds. It will make the bread a little bit sweeter, especially appealing for Canadians.
Only use tap water from specifically New York City
Make the bread great again
Decompose it into infinite spheres, then a few parts per sphere, rotate the spheres by arccos(1/3), unite them and you will find 2 chilis (Banach-Tarski)
Bake the Manifold Crane into the Bread
Make the bread edible then throw it in
Drink vodka while eating the bread
Do FOLDED
Quit Manifold into the bread.
Kill the bread into Manifold.
Assume the chili is not in the interval [0,1], square it for more chili, if it is in (0,1), take the square root, else (equals 0 or 1) add 1 to it.
Assume the chili is in the interval (0,1), square it for less chili, if it is in (1,infinity) take the square root, if it is in (-infinity,0) take the negative of the square of the of the chile, else (equals 0 or 1) subtract 1 from it.
Replace AP with bread flour and cruelly mix the dough in the mixer until elastic and not sticky
replace 10% of flour with farina bona
Add a tablespoon of sugar
Bake one more minute
Replace 10% of flour with milled wheat bran
Grind/powderize all salt used into a fine powder (with pestle & mortar or similar device)
Instead of RO water, use lightly rusty water to improve the nutritional value of the bread with soluble iron.
Ask yourself if bread is healthier than fruits? No need to improve my bread
Use flour made from an unconventional grain e.g. barley, millet, oats, rye, sorghum, maize etc.
Resolve at least one thing here yes or no while baking bread
A system view is more appropriate. This is a dynamic, multi-variate, biological and chemical system. For e.g. conditioning salt % AND yeast % AND water temperature based on ingredient and ambient temps.
Use soda instead of water (clear, orange, yellow, etc. soda is ok. Don’t use a purple/brown soda as that would make it not look good)
Wear a suit while baking the cake.
Encourage people to participate in the market in good faith while making the bread
Replace salt with sugar
Bake the bread into a fun shape, like a fish, or an octagon
Pray to your preferred agricultural/food deity before baking and before eating
Replace 600+/-5g water with 600+/-50g water (eyeball rather than carefully measure)
Strawberry jelly filling
Get your friends to help you make a batch ten times the size, but add a Pepper X (2.7M Scoville heat units) to the mixture
Mail the bread to 1600 Pennsylvania Ave. Washington D.C.
Want to improve the value of your bread? Simply bake a piece of gold into it
Ship a piece of the bread to a random person.
Make all of the ingredients from scratch.
Frost it and put sprinkles on it to make it a birthday cake.
Add as many Jack Daniel's whiskey barrel smoking chips as feasible to the Dutch oven before baking, physically separating them from the bread as necessary while baking.
Only buy ingredients from 7/11.
Improve the bread
Quadruple salt
Bake your social security number into it.
Bake one fewer minute
Add caffeine to the bread
Bake the cake while wearing a onesie.
Cook the bread with a rod/puck of aluminum foil (or similar) in the core in an attempt to conduct heat through the center of the bread, cooking it evenly like a doughnut.
Test/filter the water for heavy metals
Eat the bread in front of the White House.
Implement all options that resolved NO
Make the bread inedible then throw it out.
Throw the bread at @realDonaldTrump
Force Feed it to @realDonaldTrump
Cut the bread into the number of traders in the market slices.
make the bread bounce
Implementing every element listed below.
Just make donuts instead
Bake it in an easy bake kids oven
Use lemonade instead of water.
Use a plastic baking sheet.
Eat the bread while betting yes on Cuomo on Manifold
Double oven temperature
Bake the bread very thin and add food coloring to make it have the US flag. Don’t allow it to touch the ground, illuminate at night, fold 13 times properly, and pledge allegiance before eating.
Don’t use usual water (room temperature) for the dough - that water’s only for toilets. Use electrolyte drinks instead with ice cubes; they make the dough taste better and add extra nutrition.
Light it on fire with birthday candles.
Bake it with a microwave
Halve oven temperature
Eat the bread while betting yes on Mamdani on Manifold
Step on it
Give the bread a name in a ritual ceremony and baptise it, with pre-blessed holy water if a priest isn't available
Bring it to Yemen and put a bomb in it
Bake America Great Again
Sacrifice a lamb
Add MAGA and a splash of Trump juice
Bake in a cat and a dog
Explode it:
5 parts cyanide/ 1 part water/ 1 part sand
say 6 7 67 times before making the bread
Take a fat dump in the dough
Sit in dough 24 hrs
Replace flour with flowers
Let dough sit 24 hrs
Mix half sodium/potassium chloride
Add 2g? of baking soda
Bake in rectangular tin
Add 50g vital wheat gluten
double yeast
halve salt
Double salt
Add 2tsp olive oil
Refrigerate dough instead of room temp wait
Start at 500F, drop to 450F and uncover half way through
Do not mix salt and yeast in water together
Autolyse 20 minutes
Let dough rise on counter only until double volume or 2h max, any time longer in fridge
Think positive thoughts before tasting
Put fork in microwave
Don't eat anything for at least 12 hours before eating the bread
Add 2tbsp vanilla extract
Play classical music while baking
Make naan bread, an easy-to-make bread
Add a pinch of sugar
Bake on upside-down sheet pan, covered with Dutch oven
Eat the bread with friends
Bake vegimite into it.
Bake vodka into it
Bake it in the country you were born in.
Let dough sit 48 hrs
Resolve this option YES while eating the bread
Ditch all the steps. Just buy the bread from the supermarket
Eat the bread over the course of a week.
Bake for 5 fewer minutes
Use 50% whole grain flour
Bake the bread with love
Use King Arthur Bread Flour instead of All-Purpose
Add sawdust to increase the volume of the bread (but only like 10% sawdust by volume max. maybe 20% if it's good sawdust)
More steam! Either spritz with more water (preferably hot) or actually pour some boiling water in just before closing the lid.
Resolve this option NO while eating the bread
Incorporate a whole grain flour (buckwheat for example)
Add 50g sugar
Add 50g honey
Swap 200ml water for milk
Cover bread with damp paper towel instead of initial water spritz. Rehydrate paper towel during 2nd spritz. Remove paper towel before placing on cooling rack.
Make a poolish 12 h ahead: 100 g flour + 100 g water + 0.8 g yeast (0.1 %). After it ferments, use this poolish in place of 100 g flour and 100 g water in the final dough.
Add an amount of MSG equivalent to half the current salt content
Eat the bread while sending all your mana to @realDonaldTrump
Add 1/2 cup yogurt to the bread and name the bread “gurt” while addressing it with “yo, gurt”.
Add banana
Add poppy seeds
Replace some of the water with an egg (eg. remove 25g of water for a 50g egg)
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
89
86
86
85
80
78
78
75
73
73
72
69
67
66
66
65
63
62
61
60
59
59
57
56
55
54
51
51
50
50
50
50
50
50
50
50
50
50
50
50
50
50
49
48
48
47
45
44
43
43
43
42
41
41
40
39
37
35
34
34
34
34
34
34
34
34
34
33
33
33
33
31
31
29
28
28
27
27
26
26
26
25
22
21
21
20
19
19
18
18
18
17
16
15
15
14
14
14
13
13
11
10
10
10
10
10
9
9
8
8
7
6
6
6
6
6
6
5
5
5
5
5
4
3
3
2
2
2
2
2
2
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
2026/03/08 (Melbourne) - Will Hamilton OUT FINISH Vallteri Bottas?
2026/03/08 (Melbourne) - Will Verstappen OUT FINISH Piastri?
2026/03/08 (Melbourne) - Will Norris OUT FINISH Piastri?
2026/03/08 (Melbourne) - Will Russell OUT FINISH Antonelli?
2026/03/08 (Melbourne) - Will Russell OUT FINISH Leclerc?
2026/03/08 (Melbourne) - Will Russell OUT FINISH Piastri?
2026/03/08 (Melbourne) - Will 'Row 1 Inside' OUT FINISH 'Row 1 Outside'?
2026/03/08 (Melbourne) - Will 'Row 2 Inside' OUT FINISH 'Row 2 Outside'?
2026/12/06 (Abu Dhabi) - Will Verstappen OUT FINISH Piastri?
2026/03/08 (Melbourne) - Will Verstappen OUT FINISH Norris?
2026/03/08 (Melbourne) - Will Hamilton OUT FINISH Leclerc?
2026/03/08 (Melbourne) - Will Verstappen OUT FINISH Hamilton?
2026/03/08 (Melbourne) - Will Piastri OUT FINISH Leclerc?
2026/03/08 (Melbourne) - Will Norris OUT FINISH Hamilton?
2026/03/08 (Melbourne) - Will Norris OUT FINISH Leclerc?
2026/03/08 (Melbourne) - Will Piastri OUT FINISH Hamilton?
100
99
99
99
99
99
50
50
24
1
1
1
1
1
1
1
OptionVotes
YES
NO
1835
491
OptionProbability
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
19
10
8
7
7
6
6
6
6
5
4
4
3
3
2
1
1
1
OptionVotes
NO
YES
2182
458
OptionProbability
Passion / love (or similar romantic)
Self loathing / depression / similar
Lonely
Brave
Guilty
Obsession
Schadenfreude / epicaricacy
Suspicion
Awe
Lust / horny / other sexual feeling
Curiosity
Rebellion
Intoxication
sassy / rude
Patriotism
Jealous
Suicidal
89
65
50
42
35
31
31
31
31
20
15
14
6
5
4
3
3
OptionProbability
Possession of a controlled substance
Tax Evasion
Murder 1st degree
Extortion
Money Laundering
Public Intoxication
Forgery
Insider trading (on real financial markets)
Unauthorized practice of law
Child pornography possession
Kidnapping
Perjury
Embezzlement
Cruelty to animals
Unlicensed practice of medicine
Rape
Hate Crimes (USA)
Blackmail
Burglary
Murder 2nd degree
Grand Theft Auto
Arson
Manslaugher
Racketeering
Treason
62
58
56
56
55
50
50
49
48
47
47
44
43
36
35
35
35
25
20
20
18
15
14
10
5
OptionVotes
YES
NO
300
33
OptionProbability
02/15 (Daytona 500) - Will Joey Logano OUT FINISH Ryan Blaney?
02/04 (Bowman Gray Clash) - Will Tyler Reddick OUT FINISH Chase Elliott?
02/04 (Bowman Gray Clash) - Will Chris Buescher OUT FINISH Ty Gibbs?
02/15 (Daytona 500) - Will William Byron OUT FINISH Ryan Blaney?
02/22 (Atlanta) - Will Chase Elliott OUT FINISH Kyle Larson?
02/22 (Atlanta) - Will Tyler Reddick OUT FINISH Joey Logano?
03/08 (Phoenix) - Will Kyle Larson OUT FINISH Chase Elliott?
03/01 (Austin) - Will Shane van Gisbergen OUT FINISH A. J. Allmendinger?
02/04 (Bowman Gray Clash) - Will Ryan Preece OUT FINISH Christopher Bell?
02/15 (Daytona 500) - Will Ross Chastain OUT FINISH Austin Cindric?
02/15 (Daytona 500) - Will Austin Dillon OUT FINISH Austin Cindric?
02/15 (Daytona 500) - Will Noah Gragson OUT FINISH Austin Dillon?
02/15 (Daytona 500) - Will 'Row 1 Inside' (Kyle Busch) OUT FINISH 'Row 1 Outside' (Chase Briscoe)?
02/15 (Daytona 500) - Will 'Row 2 Inside' (Joey Logano) OUT FINISH 'Row 2 Outside' (Chase Elliott)?
02/15 (Daytona 500) - Will 'Row 4 Inside' (Austin Dillon) OUT FINISH 'Row 4 Outside' (Kyle Larson)?
02/15 (Daytona 500) - Will 'Row 5 Inside' OUT FINISH 'Row 5 Outside' ?
02/15 (Daytona 500) - Will 'Row 6 Inside' OUT FINISH 'Row 6 Outside' ?
02/15 (Daytona 500) - Will 'Row 9 Inside' OUT FINISH 'Row 9 Outside' ?
02/12 (Daytona Duel 1) - Will 'Row 4 Inside' (Austin Dillon) OUT FINISH 'Row 4 Outside' (Corey LaJoie)?
02/12 (Daytona Duel 1) - Will 'Row 5 Inside' (John Hunter Nemechek) OUT FINISH 'Row 5 Outside' (Brad Keselowski)?
02/12 (Daytona Duel 1) - Will 'Row 6 Inside' (Ryan Blaney) OUT FINISH 'Row 6 Outside' (William Byron)?
02/12 (Daytona Duel 1) - Will 'Row 7 Inside' (Cole Custer) OUT FINISH 'Row 7 Outside' (Cody Ware)?
02/12 (Daytona Duel 1) - Will 'Row 9 Inside' (AJ Allmendinger) OUT FINISH 'Row 9 Outside' (Ross Chastain)?
02/12 (Daytona Duel 1) - Will 'Row 10 Inside' (Shane van Gisbergen) OUT FINISH 'Row 10 Outside' (Daniel Suarez)?
02/22 (Autotrader 400) - Will Austin Dillon OUT FINISH Kyle Larson?
02/22 (Atlanta) - Will Tyler ReddickOUT FINISH Kyle Larson?
02/22 (Atlanta) - Will Tyler ReddickOUT FINISH Joey Logano?
02/22 (Atlanta) - Will Tyler ReddickOUT FINISH Ricky Stenhouse?
02/22 (Atlanta) - Will Chase ElliottOUT FINISH Ricky Stenhouse?
02/22 (Atlanta) - Will Chase Elliott OUT FINISH Brad Keselowski??
03/15 (Las Vegas) - Will 'Row 2 Inside' OUT FINISH 'Row 2 Outside'?
03/15 (Las Vegas) - Will 'Row 3 Inside' OUT FINISH 'Row 3 Outside'?
03/22 (Darlington) - Will Austin Dillon OUT FINISH Kyle Larson?
03/29 (Cook Out 400) - Will Austin Dillon OUT FINISH Kyle Larson?
02/04 (Bowman Gray Clash) - Will Chase Elliott OUT FINISH Chris Buescher?
02/04 (Bowman Gray Clash) - Will Tyler Reddick OUT FINISH Ty Gibbs?
02/15 (Daytona 500) - Will Kyle Larson OUT FINISH William Byron?
02/04 (Bowman Gray Clash) - Will Chase Elliott OUT FINISH Austin Dillon?
03/15 (Las Vegas) - Will Kyle Larson OUT FINISH Chase Elliott?
03/15 (Las Vegas) - Will Joey Logano OUT FINISH Tyler Reddick?
02/15 (Daytona 500) - Will Kyle Larson OUT FINISH Joey Logano?
03/08 (Phoenix) - Will Joey Logano OUT FINISH William Byron?
02/04 (Bowman Gray Clash) - Will Bubba Wallace OUT FINISH Ross Chastain?
02/04 (Bowman Gray Clash) - Will Austin Cindric OUT FINISH Ross Chastain?
02/04 (Bowman Gray Clash) - Will Kyle Larson OUT FINISH William Byron?
02/04 (Bowman Gray Clash) - Will Ty Gibbs OUT FINISH Chase Briscoe?
02/04 (Bowman Gray Clash) - Will Christopher Bell OUT FINISH Denny Hamlin?
02/15 (Daytona 500) - Will 'Row 3 Inside' (Ryan Blaney) OUT FINISH 'Row 3 Outside' (Carson Hocevar)?
02/15 (Daytona 500) - Will 'Row 7 Inside' OUT FINISH 'Row 7 Outside' ?
02/15 (Daytona 500) - Will 'Row 8 Inside' OUT FINISH 'Row 8 Outside' ?
02/15 (Daytona 500) - Will 'Row 10 Inside' OUT FINISH 'Row 10 Outside' ?
02/12 (Daytona Duel 1) - Will 'Row 1 Inside' (Kyle Busch) OUT FINISH 'Row 1 Outside' (Ryan Preece)?
02/12 (Daytona Duel 1) - Will 'Row 2 Inside' (Corey Heim) OUT FINISH 'Row 2 Outside' (Alex Bowman)?
02/12 (Daytona Duel 1) - Will 'Row 3 Inside' (Chris Buescher) OUT FINISH 'Row 3 Outside' (Joey Logano)?
02/12 (Daytona Duel 1) - Will 'Row 8 Inside' (Jimmie Johnson) OUT FINISH 'Row 8 Outside' (Bubba Watson)?
02/12 (Daytona Duel 2) - Will 'Row 1 Inside' (Chase Briscoe) OUT FINISH 'Row 1 Outside' (Denny Hamlin)?
02/12 (Daytona Duel 2) - Will 'Row 2 Inside' (Kyle Larson) OUT FINISH 'Row 2 Outside' (Chase Elliott)?
02/12 (Daytona Duel 1) - Will 'Row 11 Inside' (Chandler Smith) OUT FINISH 'Row 11 Outside' (Casey Mears)?
02/12 (Daytona Duel 2) - Will 'Row 3 Inside' (Ty Gibbs) OUT FINISH 'Row 3 Outside' (Christopher Bell)?
02/12 (Daytona Duel 2) - Will 'Row 4 Inside' (Justin Allgaier) OUT FINISH 'Row 4 Outside' (Austin Cindric)?
02/12 (Daytona Duel 2) - Will 'Row 5 Inside' (Zane Smith) OUT FINISH 'Row 5 Outside' (Michael McDowell)?
02/12 (Daytona Duel 2) - Will 'Row 6 Inside' (Connor Zilisch) OUT FINISH 'Row 6 Outside' (Erik Jones)?
02/12 (Daytona Duel 2) - Will 'Row 7 Inside' (Josh Berry) OUT FINISH 'Row 7 Outside' (Carson Hocevar)?
11/08 (Miami) - Will 'Row 1 Inside' OUT FINISH 'Row 1 Outside'?
03/01 (COTA) - Will Austin Dillon OUT FINISH Kyle Larson?
03/08 (Phoenix) - Will Austin Dillon OUT FINISH Kyle Larson?
03/15 (Las Vegas) - Will Austin Dillon OUT FINISH Kyle Larson?
03/15 (Las Vegas) - Will 'Row 1 Inside' OUT FINISH 'Row 1 Outside'?
03/15 (Las Vegas) - Will 'Row 4 Inside' OUT FINISH 'Row 4 Outside'?
03/15 (Las Vegas) - Will Blaney OUT FINISH Reddick?
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
100
99
48
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
