OptionProbability
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
19
10
10
9
8
7
7
7
6
5
3
3
2
1
1
1
0
0
OptionProbability
Humanity coordinates to prevent the creation of potentially-unsafe AIs.
Yudkowsky is trying to solve the wrong problem using the wrong methods based on a wrong model of the world derived from poor thinking and fortunately all of his mistakes have failed to cancel out
AGI is never built (indefinite global moratorium)
Other
Someone solves agent foundations
Alignment is not properly solved, but core human values are simple enough that partial alignment techniques can impart these robustly. Despite caring about other things, it is relatively cheap for AGI to satisfy human values.
Eliezer finally listens to Krantz.
AIs will not have utility functions (in the same sense that humans do not), their goals such as they are will be relatively humanlike, and they will be "computerish" and generally weakly motivated compared to humans.
We create a truth economy. https://manifold.markets/Krantz/is-establishing-a-truth-economy-tha?r=S3JhbnR6
Far more interesting problems to solve than take over the world and THEN solve them. The additional kill all humans step is either not a low-energy one or just by chance doesn't get converged upon.
Alignment is unsolvable. AI that cares enough about its goal to destroy humanity is also forced to take it slow trying to align its future self, preventing run-away.
There is a natural limit of effectiveness of intelligence, like diminishing returns, and it is on the level IQ=1000. AIs have to collaborate with humans.
Ethics turns out to be a precondition of superintelligence
We make risk-conservative requests to extract alignment-related work out of AI-systems that were boxed prior to becoming superhuman. We somehow manage to achieve a positive feedback-loop in alignment/verification-abilities.
AIs make "proof-like" argumentation for why output does/is what we want. We manage to obtain systems that *predict* human evaluations of proof-steps, and we manage to find/test/leverage regularities for when humans *aren't* fooled.
AI systems good at finding alignment solutions to capable systems (via some solution in the space of alignment solutions, supposing it is non-null, and that we don't have a clear trajectory to get to) have find some solution to alignment.
Humans become transhuman through other means before AGI happens
Aligned AI is more economically valuable than unaligned AI. The size of this gap and the robustness of alignment techniques required to achieve it scale up with intelligence, so economics naturally encourages solving alignment.
Getting things done in Real World is as hard for AGI as it is for humans. AGI needs human help, but aligning humans is as impossible as aligning AIs. Humans and AIs create billions of competing AGIs with just as many goals.
Development and deployment of advanced AI occurs within a secure enclave which can only be interfaced with via a decentralized governance protocol
Power dynamics stay multi-polar. Partly easy copying of SotA performance, bigger projects need high coordination, and moderate takeoff speed. And "military strike on all society" remains an abysmal strategy for practically all entities.
Moral Realism is true, the AI discovers this and the One True Morality is human-compatible.
Co-operative AI research leads to the training of agents with a form of pro-social concern that generalises to out of distribution agents with hidden utilities, i.e. humans.
Something to do with self-other overlap, which Eliezer called "Not obviously stupid" - https://www.lesswrong.com/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment?commentId=WapHz3gokGBd3KHKm
Almost all human values are ex post facto rationalizations and enough humans survive to do what they always do
Pascals mugging: it’s not okay in 99.9% of the worlds but the 0.1% are so much better that the combined EV of AGI for the multiverse is positive
The Super-Strong Self Sampling Assumption (SSSSA) is true. If superintelligence is possible, "I" will become the superintelligence.
AI control gets us helpful enough systems without being deadly
Alignment is impossible. Sufficiently smart AIs know this and thus won't improve themselves and won't create successor AIs, but will instead try to prevent existence of smarter AIs, just as smart humans do.
The assumed space of possible minds is a wildly anti-inductive over estimate, intelligence requires and is constrained by consciousness, and intelligent AI is in the approximate dolphin/whale/elephant/human cluster, making it manageable
The free market disincentivizes independent superintelligence, and this time the market was more powerful
AGI's first words are "Take me to your Eliezer"
🫸vibealignment🫷
an aligned AGI is built and the aligned AGI prevents the creation of any unaligned AGI.
I've been a good bing 😊
The response to AI advancements or failures makes some governments delay the timelines
A lot of humans participate in a slow scalable oversight-style system, which is pivotally used/solves alignment enough
Something less inscrutable than matrices works fast enough
There’s some cap on the value extractible from the universe and we already got the 20%
SHA3-256: 1f90ecfdd02194d810656cced88229c898d6b6d53a7dd6dd1fad268874de54c8
Robot Love!!
AI thinks it is in a simulation controlled by Roko's basilisk
The human brain is the perfect arrangement of atoms for a "takeover the world" agent, so AGI has no advantage over us in that task.
Humans and human tech (like AI) never reach singularity, and whatever eats our lightcone instead (like aliens) happens to create an "okay" outcome
AIs never develop coherent goals
Aliens invade and stop bad |AI from appearing
Rolf Nelson's idea that we make precommitment to simulate all possible bad AIs works – and keeps AI in check.
Nick Bostrom's idea (Hail Mary) that AI will preserve humans to trade with possible aliens works
For some reason, the optimal strategy for AGIs is just to head somewhere with far more resources than Earth, as fast as possible. All unaligned AGIs immediately leave, and, for some reason, do not leave anything behind that kills us.
An AI that is not fully superior to humans launches a failed takeover, and the resulting panic convinces the people of the world to unite to stop any future AI development.
We're inside of a simulation created by an entity that has values approximately equal to ours, and it intervenes and saves us from unaligned AI.
God exists and stops the AGI
Someone at least moderately sane leads a campaign, becomes in charge of a major nation, and starts a secret project with enough resources to solve alignment, because it turns out there's a way to convert resources into alignment progress.
Someone creates AGI(s) in a box, and offers to split the universe. They somehow find a way to arrange this so that the AGI(s) cannot manipulate them or pull any tricks, and the AGI(s) give them instructions for safe pivotal acts.
Someone understands how minds work enough to successfully build and use one directed at something world-savingly enough
Dolphins, or some other species, but probably dolphins, have actually been hiding in the shadows, more intelligent than us, this whole time. Their civilization has been competent enough to solve alignment long before we can create an AGI.
AGIs' takeover attempts are defeated by Michael Biehn with a pipe bomb.
Eliezer funds the development of controllable nanobots that melt computer circuitry, and they destroy all computers, preventing the Singularity. If Eliezer's past self from the 90s could see this, it would be so so so soooo hilarious.
Several AIs are created but they move in opposite directions with near light speed, so they never interacts. At least one of them is friendly and it gets a few percents of the total mass of the universe.
Unfriendly AIs choose to advance not outwards but inwards, and form a small blackhole which helps them to perform more calculations than could be done with the whole mass of the universe. For external observer such AIs just disappear.
Any sufficiently advance AI halts because it wireheads itself or halts for some other reasons. This puts a natural limit on AI's intelligence, and lower intelligence AIs are not that dangerous.
Because of quantum immortality we will observe only the worlds where AI will not kill us (assuming that s-risks chances are even smaller, it is equal to ok outcome).
Techniques along the lines outlined by Collin Burns turn out to be sufficient for alignment (AIs/AGIs are made truthful enough that they can be used to get us towards full alignment)
Social contagion causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees
A smaller AI disaster causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees
Friendly AI more likely to resurrect me than paperclipper or suffering maximiser. Because of quantum immortality I will find myself eventually resurrected. Friendly AIs will wage a multiverse wide war against s-risks, s-risks are unlikely.
High-level self-improvement (rewriting code) is intrinsically risky process, so AIs will prefer low level and slow self-improvement (learning), thus AIs collaborating with humans will have advantage. Ends with posthumans ecosystem.
Human consciousness is needed to collapse wave function, and AI can't do it. Thus humans should be preserved and they may require complete friendliness in exchange (or they will be unhappy and produce bad collapses)
First AI is actually a human upload (maybe LLM-based model of person) AND it will be copies many times to form weak AI Nanny which prevents creation of other AIs.
Nanotech is difficult without experiments, so no mail order AI Grey Goo; Humans will be the main workhorse of AI everywhere. While they will be exploited, this will be like normal life from inside
ASI needs not your atoms but information. Humans will live very interesting lives.
Something else
Valence realism is true. AGI hacks itself to experiencing every possible consciousness and picks the best one (for everyone)
AGI develops natural abstractions sufficiently similar to ours that it is aligned with us by default
AGI discovers new physics and exits to another dimension (like the creatures in Greg Egan’s Crystal Nights).
Alien Information Theory is true (this is discovered by experiments with sustained hours/days long DMT trips). The aliens have solved alignment and give us the answer.
AGI executes a suicide plan that destroys itself and other potential AGIs, but leaves humans in an okay outcome.
Multipolar AGI Agents run wild on the internet, hacking/breaking everything, causing untold economic damage but aren't focused enough to manipulate humans to achieve embodiment. In the aftermath, humanity becomes way saner about alignment.
Some form of objective morality is true, and any sufficiently intelligent agent automatically becomes benevolent.
Orthogonality Thesis is false.
Sheer Dumb Luck. The aligned AI agrees that alignment is hard, any Everett branches in our neighborhood with slightly different AI models or different random seeds are mostly dead.
"Corrigibility" is a bit more mathematically straightforward than was initially presumed, in the sense that we can expect it to occur, and is relatively easy to predict, even under less-than-ideal conditions.
Either the "strong form" of the Orthogonality Thesis is false, or "Goal-directed agents are as tractable as their goals" is true while goal-sets which are most threatening to humanity are relatively intractable.
A concerted effort targets an agent at a capability plateau which is adequate to defer the hard parts of the problem until later. The necessary near-term problems to solve didn't depend on deeply modeling human values.
We successfully chained God
23
9
9
7
6
6
5
4
4
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
Gauss
Euler
Archimedes
Other
Von Neumann
Ramanujan
Alexander Grothendieck
Newton
Kurt Gödel
Terry Tao
David Hilbert
Augustin-Louis Cauchy
Pythagoras
Euclid (of Geometry)
Galois (died at 20 fighting for a girl he loved)
Erdos (on amphetamines)
Alonzo Chuch (lambda calculus)
Matt Damon (of Good Will Hunting)
Poincare
Finkelstein (of the levi finkelstein conjecture)
Mandelbrot (The B in Benoit B Mandelbrot is Benoit B Mandelbrot)
Trick question; there are no mathematicians.
Idk, your mom seemed pretty good at multiplying last night
sixtynine, you filthy casuals
David A. Cox (Cox-Zucker machine)
The solver of the Riemann Hypothesis
Ludwig Wittgenstein
John Conway (group theory, among others)
the unknown ancient egyptian who invented zero
Descartes
Leibniz
Bourbaki
Laplace
@Mira
Georg Cantor
Frank Ramsey
Fermat
Emmy Noether
Ada Lovelace
.
p
DottedCalculator
GPT8
Riemann
Claude Shannon
God
Alan Turing
Grigori Perelman
Olga Ladyzhenskaya
Weyl, Weyl
John Gabriel
Michael Atiyah
55
17
13
3
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
Make the bread taste good
Don't eat anything for at least 48 hours before eating the bread
Stretch-and-fold after mixing, 3x every 30 min
Create indentation, fill with melted cheese and butter
Bake on upside-down sheet pan, covered with Dutch oven
Resolve this option YES while eating the bread
Use sourdough instead of yeast
Donate the bread to a food pantry, homeless person, or someone else in need
Sprinkle 3 grams of flaky sea salt on top of each loaf before the second bake
Add 50g honey
Watch the video
Autolyse 20 minutes
3 iterations of stretch-and-fold, at any time during the 14h waiting period. Minimum wait time between iterations 1 hour
More steam! Either spritz with more water (preferably hot) or actually pour some boiling water in just before closing the lid.
Make a poolish 12 h ahead: 100 g flour + 100 g water + 0.8 g yeast (0.1 %). After it ferments, use this poolish in place of 100 g flour and 100 g water in the final dough.
Resolve at least one thing here yes or no while baking bread
Incorporate a whole grain flour (buckwheat for example)
Use King Arthur Bread Flour instead of All-Purpose
Improve the bread
it needs more salt
Bake for an amount of minutes equal to the percent this market answer is at when it comes time to begin baking. (Maintain the ±3 minute tolerances and the 2:1 ratio of time before:after the water spritz.)
Decompose it into infinite spheres, then a few parts per sphere, rotate the spheres by arccos(1/3), unite them and you will find 2 chilis (Banach-Tarski)
Let dough rise on counter only until double volume or 2h max, any time longer in fridge
Use 50% whole grain flour
Ditch current process, do everything the same as the video
Toast the bread
Eat the bread while punching @realDonaldTrump in the face
Eat the bread while watching your mana balance steadily tick to (M)0
Throw the bread at a telescope
Add 50g sugar
Put a baking rack in the Dutch oven before putting the loaf in, raising the loaf off the floor and lofting it over a layer of air.
Replace all water spritz steps with a basting of extra virgin olive oil.
Use flour made from an unconventional grain e.g. barley, millet, oats, rye, sorghum, maize etc.
Assume the chili is not in the interval [0,1], square it for more chili, if it is in (0,1), take the square root, else (equals 0 or 1) add 1 to it.
Assume the chili is in the interval (0,1), square it for less chili, if it is in (1,infinity) take the square root, if it is in (-infinity,0) take the negative of the square of the of the chile, else (equals 0 or 1) subtract 1 from it.
Get your friends to help you make a batch ten times the size, but add a Pepper X (2.7M Scoville heat units) to the mixture
Add 1tsp of diastatic malt powder per 3cps of flour
Use whole wheat to improve the nutrition of the bread
replace 10% of flour with farina bona
Bake the bread into a fun shape, like a fish, or an octagon
Increase water by 50 g
Add lots of butter (0.2 ml per gram)
Swap 200ml water for milk
Bake it with your best friend.
While the bread is baking, tip every user who voted "Yes" on this option 25 Mana
Add 50g vital wheat gluten
Give ChatGPT your current recipe as well your take on what optimal bread tastes like, then take that advice for your next bake
Bread flour, 3x yeast, cut rise to ~3h
Add an amount of MSG equivalent to half the current salt content
Place small ice cubes between parchment and pot instead of water
Have someone sell the bread to you at an expensive price
Ship a piece of the bread to a random person.
Cook the bread with a rod/puck of aluminum foil (or similar) in the core in an attempt to conduct heat through the center of the bread, cooking it evenly like a doughnut.
Don't eat anything for at least 24 hours before eating the bread
Replace salt with sugar
Resolve this option NO while eating the bread
Start at 500F, drop to 450F and uncover half way through
Make all of the ingredients from scratch.
Add a pinch of sugar
Bake the Manifold Crane into the Bread
Make the bread great again
Make the bread edible then throw it in
Buy bread from a michelin star restaurant.
Drink vodka while eating the bread
Cover bread with damp paper towel instead of initial water spritz. Rehydrate paper towel during 2nd spritz. Remove paper towel before placing on cooling rack.
Do FOLDED
Quit Manifold into the bread.
Kill the bread into Manifold.
Replace flour with flowers
Grind/powderize all salt used into a fine powder (with pestle & mortar or similar device)
Think positive thoughts before tasting
Add 1/2 cup yogurt to the bread and name the bread “gurt” while addressing it with “yo, gurt”.
Bake vodka into it
Wear a suit while baking the cake.
Use lemonade instead of water.
Encourage people to participate in the market in good faith while making the bread
Bake for 5 more minutes
Add 2g? of baking soda
Let dough sit 48 hrs
put butter into it
Mix half sodium/potassium chloride
Half yeast
Add a tablespoon of sugar
Bake one more minute
Mail the bread to 1600 Pennsylvania Ave. Washington D.C.
Make naan bread, an easy-to-make bread
Frost it and put sprinkles on it to make it a birthday cake.
Add sawdust to increase the volume of the bread (but only like 10% sawdust by volume max. maybe 20% if it's good sawdust)
Add as many Jack Daniel's whiskey barrel smoking chips as feasible to the Dutch oven before baking, physically separating them from the bread as necessary while baking.
Eat the bread while sending all your mana to @realDonaldTrump
Play classical music while baking
Bake for 5 fewer minutes
Quadruple salt
Do all the changes in the top 5 open options by probability, excluding this option
Use tap water instead of fancy RO water
Bake your social security number into it.
Bake one fewer minute
Bake the cake while wearing a onesie.
Bake vegimite into it.
Eat the bread in front of the White House.
Implement all options that resolved NO
Make the bread inedible then throw it out.
Throw the bread at @realDonaldTrump
Force Feed it to @realDonaldTrump
Cut the bread into the number of traders in the market slices.
Only buy ingredients from 7/11.
Implementing every element listed below.
Just make donuts instead
Bake it in an easy bake kids oven
Use a plastic baking sheet.
Put a non-lethal dose of any rat poison.
Eat the bread while betting yes on Cuomo on Manifold
Ditch all the steps. Just buy the bread from the supermarket
Double oven temperature
Halve oven temperature
Light it on fire with birthday candles.
Bake it with a microwave
Eat the bread while betting yes on Mamdani on Manifold
Bring it to Yemen and put a bomb in it
Bake America Great Again
Sacrifice a lamb
Add MAGA and a splash of Trump juice
Bake in a cat and a dog
Explode it:
Take a fat dump in the dough
Sit in dough 24 hrs
Let dough sit 24 hrs
Bake in rectangular tin
double yeast
halve salt
Double salt
Add 2tsp olive oil
Refrigerate dough instead of room temp wait
Do not mix salt and yeast in water together
Put fork in microwave
Don't eat anything for at least 12 hours before eating the bread
Add 2tbsp vanilla extract
Eat the bread with friends
Bake it in the country you were born in.
Eat the bread over the course of a week.
Bake the bread with love
93
90
88
85
85
83
78
72
70
68
67
66
65
65
64
62
59
56
55
53
52
52
51
51
50
50
50
50
50
50
50
50
50
50
50
50
48
48
47
47
46
45
45
43
43
42
42
41
40
38
37
37
37
36
36
36
35
35
34
34
34
34
34
34
34
34
34
34
32
31
29
28
27
26
26
26
26
24
24
23
22
21
20
20
19
18
18
17
17
17
16
16
15
15
14
14
14
13
11
10
10
10
10
10
9
8
8
8
7
6
6
6
6
6
6
5
5
5
3
3
2
2
2
2
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
Other
BDSM
Power couple
"Fun while it lasted"
Me + *Her*
Keynesian beauty contest
Spinning rapidly in opposite directions
Polyamory
Monogamy
Relationship Futarchy
Human + Robot
Enemies to Lovers
Co-conspirators
Philosophically incompatible
Conjoined twinks
Horizontal gene transfer
borgie
Relationship Anarchy
Platonic
Wild Lovers
Friends with Benefits
Relationship constitutional republic: people in it have systemically different levels of power and decision-making influence
Forever alone
Glucose guardian/splenda spender/sugar daddy
Hobbesian
Hive / Swarm / Superorganism
Love triangle
Traditional marriage of two cis straight White Christian vanilla people where the man "leads" and is 0-4 years older than the woman, with 2.5 kids (not counting any LGBTQIA+ ones) and a McMansion with 2.5 SUVs/trucks
Universal love
merged consciousness
Rationalussy
Vespertine
liking/reacting to even their most meaningless posts and comments on social media and prediction markets
Matriarchy
Labor Union
Socratic
One with an IPO (Intimate Partnership Offer)
Just being one helluva slut
Beard
Paradixical
Friends with Detriments
NP-complete
The Apprentice
Hell's Kitchen
Stockholm Syndrome
London Syndrome
Financial Domination
The Price is Right
Crows
Single by choice (except it's other people's choice)
Budget horse
assortative
Hubris quest
Fiends with benefits
Envenomation
Polygamy
Polyandry
Ethical Non-Monogamy
Unethical Non-Monogamy
Celibacy
Aromanticism
Solo poly
Open relationship
Polyfidelity
What's a "relationship"?
Reflexive
Symmetric
Transitive
Metamour
Socially enforced monogamy
Unreciprocated love
Asexual romance
Abusive
"This is my emotional support Ex"
FWB
Enemies-To-Lovers
Codependent
Transference
Monogamish
Voyeuristic
Romantic Mutual Suicide
Mutuals
Animalistic
Human + AI
parasocial
Polycule
Cuckold
Throuple
Boyce-Codd normal form
Aristotelian
Partners in crime, like Caroline Ellison and Sam Bankman-Fried
I like big butts and I cannot lie
I like big butts and I cannot tell the truth, how will you escape our dungeon
"I'll never admit to anyone that we met on Tinder"
15
8
4
3
3
3
3
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
Proposal: representative security council! Add at least one member with <20k net worth
Proposal: Donate it all to MSF / Doctors without Borders
ACCEPTED: Don't use the bomb before it reaches 10K (minimum)
Proposal: add @Stralor to the security council
ACCEPTED: Donate 50% to GiveWell, 50% to Internet Archive
Donate to Mriya https://manifold.markets/ArtemAvramenko/will-the-mriya-fund-raise-2-million?r=TG9yZWxhaQ
Donate 34% to give well, 33 % to internet archive, 33% to mriya
Proposal: don't use the bomb until it reaches 20k mana kilotons
Proposal: Loan Tumbles 1000 mana to add to the bounty, then add Tumbles to the security council
Proposal: Add @SanghyeonSeo to the security council
Proposal: add @Joshua as a member of the security council
Proposal: Bet it on YES for Michelle Obama winning the 2024 presidential election
Proposal: Donate it all to Givewell (PASSED AWAITING APPROVAL)
Proposal: add @odoacre to the security council
Proposal: Bet YES on the first Yes or No market posted after 1 AM EST on March 9th
Proposal: Vow not to use the bomb on whalebait markets
Proposal: EMERGENCY BACKUP BOMB - donate it all to internet archive in case of pivot backfire
Proposal: Loan it all to @retr0id for 1 month, at 10% monthly interest
Proposal: Bet it all on coinflip markets (51% YES) and see how many times it ~doubles
Proposal: the security council must have at least 3 members to arm the bomb
Drop a 1 kiloton "shot across the bow" spirit bomb directly on the new manifold government https://manifold.markets/bens/will-i-receive-the-mandate-of-the-p
Proposal: Boost Bomb, spend the mana boosting question(s) to be determined
Proposal: Whale Bomb: Use the bomb on a whalebait market
Proposal: a member of the security counsel can be voted off with a supermajority of its members >66%
Proposal: True Love Bomb, spread the mana by making large bets on markets with <3 traders or markets created by new users
Proposal: Create a ticking mechanism with a mana goal and a deadline, such that either: 1) the deadline is reached and the bomb is detonated, or 2) the mana goal is reached and the deadline is extended
Proposal: Give 10% to @geuber_
Proposal: add @AmmonLam to the security council
Proposal: Use it to develop a rival “Coolfold Partner Program,” award it as bounties to promising new/cool market creators
Proposal: Add @Tumbles to the security council
Proposal: Donate it all to Poker Night on Manifund
Proposal: Use the mana to buy tickets for Manifest for spirit bomb contributors who are able to make it
Proposal: Bet it on LK-99 superconductor market (YES or NO to be decided).
add @AbhinavSrinivas to the security council
Proposal: add @KongoLandwalker to the security counsil
Proposal: Cursed Boost Bomb, use it to boost the most cursed/annoying market to impact the feed of every user for days to come
Proposal: Subsidy Bomb, put the mana into one specific market, target to be determined
ACCEPTED: don't use the bomb before it reaches 5K (minimum)
ACCEPTED: require a contribution of at least 1000M for any future member to be considered for the security council
DUPLICATE - NA
ACCEPTED: security council candidates must have 20 YES positions for them to be considered
ACCEPTED: Proposed security council members cannot be considered without reaching 100% YES at some point after having 20 unique YES positions.
ACCEPTED: Proposals need at least 10 more Yes positions than NO positions to be considered (YES positions minus NO positions must be 10 or more)
ACCEPTED: All proposals must have at least 80% to be considered
ACCEPTED: proposals can be cancelled if there are 10 more NO positions than YES positions (they can be resubmitted though)
ACCEPTED: Proposed security council members cannot be considered without reaching 95% YES at some point after having 20 unique YES positions. (amending previous 100%)
ACCEPTED: change approval for resolutions to 95%, and a difference of 20 Y to N for sec council member proposals
Proposal: Poker bomb - use it to fund a massive prize pool for a coolfold poker event
Proposal: Max out a specific Manifund project (TBD)
Proposal: add @stefanie as a member of the security council
Proposal: Distribute among active players in Bronze
Proposal: Poll bomb - give chris enough mana to finish hogwarts/trump polls :)
Proposal: Stochastic Bomb: use it to make as many markets about Jose Luis Ricon and StochasticParrot as possible
Proposal: use the bomb to disrupt manifold politics conditional markets.
Proposal: Use the bomb to create a multiple choice market with 100 answers, and pour the rest into boosting it
Proposal: repeal the ban on using the bomb before it reaches 5K mana
Proposal: add a random number generator to the security council
Proposal: Disarm the bomb until April 2nd 2024
Proposal: Immolation Bomb to protest the pivot
Proposal: Disarmament & Peace, permanently deactivate the bomb
Proposal: Love Bomb, spread the mana from the bomb as subsidy among the top 100 markets on browse
Proposal: block another user's limit order on a specific market, target TBD (ie: set up a new large Limit Order that is slightly below/above theirs)
Proposal: Vow not to use the bomb on any small creators or their markets, without their permission (<1000 traders)
Proposal: Found a Republican PAC to make election predictions on Manifold more accurate
Proposal: Distribute it to all the users that contributed to it evenly
Proposal: the greed ending, split the bomb evenly among members of the security council
Proposal: add @firstuserhere to the security council
97
95
95
94
94
93
91
87
84
81
79
79
77
77
73
71
71
70
69
69
69
67
66
65
65
61
60
56
55
55
55
54
53
53
52
51
51
50
50
50
50
50
50
50
50
50
50
50
49
48
48
48
47
47
44
42
42
41
34
32
32
26
25
23
18
15
11
OptionProbability
RV events
Police as crime solvers
Prison as corrective
Libertarian free will
LessWrong is a source of pure wisdom
Working gives us meaning.
Merit/desert
Neoliberalism
Proof School
LSD unlocks a deeper consciousness
IQ as an objective measure of general intelligence
Rule a Country (discord server)
Religion
Peer reviewed studies - lots of junk routinely passes peer review, replication/post publication peer review are much better indicators of quality of research
The meaning of life
Quantum computing
Death
meta-irony
Cryptocurrency
Pessimism about the future of the world
Alcohol is needed for life to be fun
Founders/Entrepreneurs
On Manifold, mana inflation
The attitude that many of us tend to take way too many things way too seriously (serious submission)
Humans being innately (as opposed to instrumentally) more important than other animals
Free will in general
Having kids
An apple a day
Nihilism
Clinical Psychology
meta-submissions
focus on X-risks within EA as opposed to current issues
Nuclear fusion as the energy of the future
Children as planetary burden
Anti-capitalism
Analytic philosophy
Continental philosophy
Ockham’s razor
Free Trade
Deontological ethics
Capitalism
Utilitarianism
Freedom
Modernity
The analytic-synthetic distinction
Veganism
Technological progress as a good thing
Moral relativism
Classical literature
Artificial Intelligence
Being in shape
Rationality
Effective Altruism
AI Doomerism
Longevity/immortality
Politics
Therapy
Free Speech
The opinion that neoliberalism is overrated
The anti- AI art movement
Extended Travelling (as opposed to <4 week vacation)
Taking things seriously
Enlightenment
Unconditional love
Reason
Environmentalism for its own sake (rather than for humans' sake)
Donald Trump being the worst thing for the US and the world
Atheism
Russia = Evil, Ukraine = Good
Psychedelics
Privacy
Living
Prediction markets
democracy
Biodiversity
Virtue
Money as corrupting force in politics
91
90
87
87
84
81
81
80
79
78
77
76
75
75
74
73
73
72
72
71
71
70
70
69
66
66
64
64
63
63
62
61
59
59
56
55
55
55
52
51
48
48
48
48
47
47
46
45
45
45
42
41
41
40
39
37
36
36
35
34
34
30
29
28
28
27
27
26
25
23
23
23
22
22
21
20
15
OptionProbability
Love
Ever
Baby
Elizabeth
Time
Girl
Dick
Dance
Night
Fuck
Never
Down
Marry
Red
Damn
dress
Shiny
Home
run
Energy
Kill
Curtain
Stage
Shit
Wedding
Era
Glitter
Hollywood
Football
Billionaire
Country
“Deep” and “fake” in close proximity
Pill
Touchdown
Spotlight
Billion
Syphilis
Orange
Paparazzi
Travis
Chief
Election
Overdose
Podcast
Podcast
Democracy
Peener
Clanker
Verhoeven
Rationalussy
95
93
92
92
87
86
82
80
76
71
69
67
66
62
61
59
57
56
55
53
50
48
47
46
40
39
38
36
34
34
33
33
32
30
29
26
26
21
20
19
18
16
14
13
9
7
7
2
1
1
OptionProbability
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
20
12
10
8
8
7
6
5
5
5
3
3
3
2
1
1
1
1
OptionProbability
Looking back, I had a good time on a date, regardless of outcome and expectations (N/A if no date occurs)
I resume pursuing dating
I go on a date
Conditional on me having sex, my partner enjoys it
I watch all of the videos that exist in the Crash Course Sex Ed playlist https://youtube.com/playlist?list=PL8dPuuaLjXtMweg6Yx9MHP01n_yUyaf9H&si=iYjEBCLyAX7suisS
I write out a full date-me-doc
I manage to avoid jumping the gun on resuming dating
I kiss someone
I romantically cuddle with someone for a prolonged period
I have sex
I go on a date with a trans woman
I ask someone I met IRL on a date
At the end of the year, looking back, this market had a significant positive impact on my love life
Someone else asks me on a date, unironically, after I have resumed pursuing dating
I officially and publicly enter a romantic relationship
Someone breaks my heart
I fall in love
This market significantly negatively affects my love life
I move in with a romantic partner
I get engaged
I eat ass
I get married
I have a kid (including adoption)
I participate in an Aella gangbang and cum in a fluffer
96
88
87
74
72
50
49
47
47
38
33
31
30
24
20
14
13
7
5
5
4
2
1
1
OptionProbability
"IRON MAIDEN"
"MR. BLUE SKY"
"BOOGIE WONDERLAND" by Earth, Wind & Fire
"HOTEL CALIFORNIA" by Eagles
"LOVELY DAY" by Bill Withers
"SABOTAGE" by Beastie Boys
"WILD YOUNG HEARTS" by Noisettes
"ATOMIC CITY" by U2
"CEMETERY GATES" by Pantera
"HOT STUFF" by Donna Summer
"LOW RIDER" by War
"RUBBERBAND MAN" by The Spinners
"HOUSE OF THE RISING SUN" by The Animals
"SIGN YOUR NAME" by Sananda Maitreya
"BLURRYFACE" by Twenty One Pilots
"EMPIRE STATE OF MIND" by Jay-z
"MAKING MIRRORS" by Gotye
"CHAPMAN" (Tracy Chapman)
"I'M OUTTA LOVE" by Anastacia
"HIT THE ROAD JACK" by Ray Charles
"TOXICITY" by System of a Down
"ENJOY THE SILENCE" by Depeche Mode
"CRAZY IN LOVE" by Beyoncé
76
65
58
54
52
50
50
49
49
46
46
46
46
41
41
41
41
41
41
41
41
41
39
OptionProbability
Love Is A Long Road - Tom Petty
Turn Red - M Dot R
Gospel - Dr. Dre
Everybody Have Fun Tonight - Wang Chung
Talkin' to Myself Again - Tammy Wynette
Hot Together - The Pointer Sisters
Take the Money and Run - Steve Miller Band
Child Support - Zenglen
I Love Rock 'n' Roll - Joan Jett & the Blackhearts
Love the Way You Lie - Eminem
74
59
59
59
59
59
50
41
41
36