DeepNewz, mobile.
People-sourced. AI-powered. Unbiased News.
OptionProbability
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
19
18
16
10
9
7
6
5
3
2
2
1
1
1
0
0
0
0
OptionProbability
Citizen Kane
Schindler's List
The Godfather
Casablanca
Raging Bull
Singin in the Rain
Vertigo
The Shawshank Redemption
Gone with the Wind
The Wizard of Oz
City Lights
The Sound of Music
West Side Story
Lawrence of Arabia
Bridge on the River Kwai
Treasure of the Sierra Madre
King Kong (original)
Bonnie and Clyde
Midnight Cowboy
The Philadelphia Story
It Happened One Night
The Graduate
To Kill a Mockingbird
All About Eve
The Maltese Falcon
One Flew Over The Cuckoo's Nest
Dr Strangelove
Jaws
Star Wars
Psycho
2001 A Space Odyssey
Sunset Boulevard
The General
On The Waterfront
It's a Wonderful Life
Some Like it Hot
The Grapes of Wrath
ET the Extra Terrestrial
Mr Smith Goes to Washington
Double Indemnity
Apocalypse Now
Godfather Part II
Snow White and the Seven Dwarfs (original animated)
Annie Hall
A Streetcar Named Desire
Rear Window
Taxi Driver
The Deer Hunter
M*A*S*H*
North by Northwest
Rocky
The Gold Rush
Sullivan's Travels
Lord of the Rings: Fellowship of the Ring
Chinatown
High Noon
Intolerance
Nashville
Shane
Duck Soup
Network
The African Queen
Raiders of the Lost Ark
Who's Afraid of Virginia Woolf
Unforgiven
The Searchers
Saving Private Ryan
A Clockwork Orange
Cabaret
Tootsie
Do The Right Thing
American Graffiti
Pulp Fiction
12 Angry Men
Platoon
Silence of the Lambs
Titanic
The Sixth Sense
Goodfellas
Toy Story
Butch Cassidy and the Sundance Kid
In the Heat of the Night
Modern Times
The Wild Bunch
Spartacus
A Night at the Opera
Swing Time
Sophie's Choice
The Last Picture Show
The Best Years of our Lives
All the President's Men
Sunrise: A Song of Two Humans
Easy Rider
Bringing Up Baby
Yankee Doodle Dandy
The Apartment
Forrest Gump
Blade Runner
Ben-Hur
The French Connection
97
97
96
96
96
96
96
96
95
95
94
94
94
92
92
92
92
92
92
92
92
91
91
91
91
91
91
91
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
90
89
88
88
88
88
87
87
87
87
86
86
86
85
84
81
80
80
80
78
78
77
76
74
74
74
74
74
67
67
67
67
67
67
67
67
67
65
65
65
65
65
65
63
57
56
56
47
OptionProbability
Tom Holland (Peter Parker / Spider-Man)
Anthony Mackie (Sam Wilson / Captain America)
Robert Downey, Jr. (Victor von Doom / Doctor Doom, Tony Stark / Iron Man)
Pedro Pascal (Reed Richards / Mister Fantastic)
Benedict Cumberbatch (Dr. Stephen Strange)
Ebon Moss-Bachrach (Ben Grimm / The Thing)
Joseph Quinn (Johnny Storm / Human Torch)
Samuel L. Jackson (Nick Fury)
Sebastian Stan (Bucky Barnes / Winter Soldier / White Wolf)
Vanessa Kirby (Sue Storm / Invisible Woman)
Simu Liu (Shang-Chi)
Paul Rudd (Scott Lang / Ant-Man)
Chris Hemsworth (Thor)
Charlie Cox (Matt Murdock / Daredevil)
Hannah John-Kamen (Ava Starr / Ghost)
Julia Louis-Dreyfus (Valentina Allegra de Fontaine)
Pom Klementieff (Mantis)
David Harbour (Alexei Shostakov / Red Guardian)
Danny Ramirez (Joaquin Torres / Falcon)
Iman Vellani (Kamala Khan / Ms. Marvel)
Mark Ruffalo (Bruce Banner / Hulk)
Dominique Thorne (Riri Williams / Ironheart)
Florence Pugh (Yelena Belova / Black Widow)
Benedict Wong (Wong)
Chris Pratt (Peter Quill / Star-Lord)
Vin Diesel (Groot)
Ryan Reynolds (Wade Wilson / Deadpool)
Jeremy Renner (Clint Barton / Hawkeye)
Wyatt Russell (John Walker / U.S. Agent)
Brie Larson (Carol Danvers / Captain Marvel)
Letitia Wright (Shuri / Black Panther)
Karen Gillan (Nebula)
Tatiana Maslany (Jennifer Walters / She-Hulk)
Chris Evans (Steve Rogers / Captain America)
Hailee Steinfeld (Kate Bishop / Hawkeye)
Teyonah Parris (Monica Rambeau)
Xochitl Gomez (America Chavez)
Hayley Atwell (Peggy Carter / Captain Carter)
Kathryn Newton (Cassie Lang)
Bradley Cooper (Rocket Raccoon)
Will Poulter (Adam Warlock)
Tenoch Huerta Mejía (Namor)
Don Cheadle (James "Rhodey" Rhodes / War Machine)
Tessa Thompson (Valkyrie)
Winston Duke (M'Baku)
Elizabeth Olsen (Wanda Maximoff / Scarlet Witch)
Paul Bettany (Vision)
Kathryn Hahn (Agatha Harkness)
Danai Gurira (Okoye)
Oscar Isaac (Marc Spector / Moon Knight, Steven Grant / Mr. Knight, Jake Lockley)
Hugh Jackman (James "Logan" Howlett / Wolverine)
Tom Hiddleston (Loki)
Mahershala Ali (Blade)
Sean Gunn (Kraglin)
Alaqua Cox (Maya Lopez / Echo)
Lupita Nyong'o (Nakia)
Wil Deusner (Skaar)
Gemma Chan (Sersi)
Kumail Nanjiani (Kingo)
Lia McHugh (Sprite)
Brian Tyree Henry (Phastos)
Barry Keoghan (Druig)
Harry Styles (Eros / Starfox)
Patton Oswalt (Pip the Troll)
Kit Harington (Dane Whitman)
Yahya Abdul-Mateen II (Simon Williams / Wonder Man)
Evangeline Lilly (Hope van Dyne / Wasp)
Seth Green (Howard the Duck)
Zoe Saldaña (Gamora)
Dave Bautista (Drax)
Angelina Jolie (Thena)
Gael García Bernal (Jack Russell / Werewolf by Night)
Tobey Maguire (Peter Parker / Spider-Man)
Scarlett Johansson (Natasha Romanoff / Black Widow)
Andrew Garfield (Peter Parker / Spider-Man)
Lauren Ridloff (Makkari)
Emilia Clarke (G'iah)
Ralph Ineson (Galactus)
Jeffrey Wright (The Watcher)
May Calamawy (Layla El-Faouly / Scarlet Scarab)
Harrison Ford (Thaddeus "Thunderbolt" Ross)
Brett Goldstein (Hercules)
Don Lee (Gilgamesh)
Olga Kurylenko (Antonia Dreykov / Taskmaster)
94
92
91
91
90
90
89
88
88
88
87
86
85
85
84
84
83
83
83
82
81
81
79
79
78
78
78
78
78
76
76
75
75
73
73
72
72
72
70
69
69
69
69
66
66
65
65
62
59
59
55
54
54
52
50
50
50
50
50
50
50
50
50
50
50
50
48
46
41
41
41
41
37
34
34
34
34
34
32
29
23
17
16
14
OptionProbability
K. Somebody discovers a new AI paradigm that's powerful enough and matures fast enough to beat deep learning to the punch, and the new paradigm is much much more alignable than giant inscrutable matrices of floating-point numbers.
I. The tech path to AGI superintelligence is naturally slow enough and gradual enough, that world-destroyingly-critical alignment problems never appear faster than previous discoveries generalize to allow safe further experimentation.
C. Solving prosaic alignment on the first critical try is not as difficult, nor as dangerous, nor taking as much extra time, as Yudkowsky predicts; whatever effort is put forth by the leading coalition works inside of their lead time.
Something wonderful happens that isn't well-described by any option listed. (The semantics of this option may change if other options are added.)
A. Humanity successfully coordinates worldwide to prevent the creation of powerful AGIs for long enough to develop human intelligence augmentation, uploading, or some other pathway into transcending humanity's window of fragility.
B. Humanity puts forth a tremendous effort, and delays AI for long enough, and puts enough desperate work into alignment, that alignment gets solved first.
D. Early powerful AGIs realize that they wouldn't be able to align their own future selves/successors if their intelligence got raised further, and work honestly with humans on solving the problem in a way acceptable to both factions.
M. "We'll make the AI do our AI alignment homework" just works as a plan. (Eg the helping AI doesn't need to be smart enough to be deadly; the alignment proposals that most impress human judges are honest and truthful and successful.)
O. Early applications of AI/AGI drastically increase human civilization's sanity and coordination ability; enabling humanity to solve alignment, or slow down further descent into AGI, etc. (Not in principle mutex with all other answers.)
E. Whatever strange motivations end up inside an unalignable AGI, or the internal slice through that AGI which codes its successor, they max out at a universe full of cheerful qualia-bearing life and an okay outcome for existing humans.
F. Somebody pulls off a hat trick involving blah blah acausal blah blah simulations blah blah, or other amazingly clever idea, which leads an AGI to put the reachable galaxies to good use despite that AGI not being otherwise alignable.
J. Something 'just works' on the order of eg: train a predictive/imitative/generative AI on a human-generated dataset, and RLHF her to be unfailingly nice, generous to weaker entities, and determined to make the cosmos a lovely place.
H. Many competing AGIs form an equilibrium whereby no faction is allowed to get too powerful, and humanity is part of this equilibrium and survives and gets a big chunk of cosmic pie.
G. It's impossible/improbable for something sufficiently smarter and more capable than modern humanity to be created, that it can just do whatever without needing humans to cooperate; nor does it successfully cheat/trick us.
L. Earth's present civilization crashes before powerful AGI, and the next civilization that rises is wiser and better at ops. (Exception to 'okay' as defined originally, will be said to count as 'okay' even if many current humans die.)
N. A crash project at augmenting human intelligence via neurotech, training mentats via neurofeedback, etc, produces people who can solve alignment before it's too late, despite Earth civ not slowing AI down much.
You are fooled by at least one option on this list, which out of many tries, ends up sufficiently well-aimed at your personal ideals / prejudices / the parts you understand less well / your own personal indulgences in wishful thinking.
If you write an argument that breaks down the 'okay outcomes' into lots of distinct categories, without breaking down internal conjuncts and so on, Reality is very impressed with how disjunctive this sounds and allocates more probability.
20
10
8
7
6
6
6
6
6
4
4
4
3
2
2
1
1
1
OptionProbability
Isaac likes experimenting
You'd say you're more of a dog person than a cat person
You've been in a desert once
You regret ever making WvM (feel free to n/a if you'd rather not say)
You have ever looked through list of a Manifold user's bets to determine their position on some question
You've been vegetarian for >1 continuous year
(if you're not married) you've thought about marrying your current partner (if you're in a relationship)
You like listening to violin more than you like listening to piano
You're older than me (I'm 23)
you consider your partner to be your best friend as well
You're younger than 30
You think it's fine for minors to date adults
You consider yourself part of the rationality sphere
You think of yourself as a dog person more than a cat person
You seem to enjoy indie stuff more than the average person does
you've defected on some sort of deal you made with another person
The first book you remember reading in your life has an animal as its main character
You tend to wear the same outfit every day.
You think the effective altruism philosophy is good but the community overall is ineffective in implementing that philosophy
You've spent a day in which you read a (single) book (or similar) for 10+ hrs
You're an effective altruist
You are qorrenqial
You've read the CFAR Handbook
You've held a gun in your hands before
You've fired a gun at a target before
You've jumped out of a plane
You have attended some program or camp organised by MIRI
You've knowingly misresolved a market once even though you didn't get called out for it
You've been to a nightclub
You've passed out from drinking alcohol
You like to eat out more than you like to cook and eat
You've worked with LED strip lights
You've wondered what it'd be like to have a different name
*You* have more than 6 stuffed animals
You know >4 programming langauges
You know >= 3 natural languages (in at least one of reading, speaking, or hearing understanding)
You've felt desires to purchase a car with mana
You have a political belief that'd cause at least one close friend to cut ties if they knew about it
You wish you spent significantly less time on Manifold
You sleep more during 7am->7pm hours than during 7pm->7am hours
You prefer if these answers start with "You" than with "isaac"
You feel you're the smartest member of your close family (mother, father, siblings)
You have thought about shooting lasers from your eyes at traffic lights
You have once made a joke about your name's similarity with Martin Luther King Jr.
You think the effective altruism community is ineffective at PR and image management
You have done some coding in python
100
100
100
100
100
100
100
100
100
100
100
90
77
70
53
49
49
41
32
30
20
8
5
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
OptionProbability
Loki
Sylvie
Mr. Fantastic
Other
Ant-Man (Lang)
Spider-Man
Hawkeye (Barton)
Captain America (Rogers)
Iron Man
Black Widow (Romanoff)
Nick Fury
War Machine
Winter Soldier
Captain America (Wilson)
Gamora
Drax
Rocket
Groot
Nebula
Vision
Ultron
Wasp (Hope)
Black Panther (T'Challa)
Black Panther (Shuri)
Wong
Mantis
Valkyrie
Captain Marvel
Yelena Belova
Shang-Chi
Sersi
Thena
Druig
Makkari
Kingo
Phastos
Phil Coulson
Daisy Johnson
Peggy Carter
Black Bolt
Daredevil
Jessica Jones
Luke Cage
Iron Fist (Rand)
Punisher
Agatha Harkness
Hawkeye (Bishop)
Moon Knight
Ms. Marvel
Monica Rambeau
She-Hulk
Ironheart
Echo
Blade
Wonder Man
Okoye
Nova
Deadpool
Wolverine
Werewolf by Night
The Invisible Woman
The Thing
The Human Torch
Thor
Hulk
Thanos
Star-Lord
Scarlet Witch
Doctor Strange
Adam Warlock
Kang
19
5
3
2
2
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
OptionVotes
NO
YES
524
96
OptionProbability
Ineffective: Any benefits are too small to be worth the drawbacks
Unproven but promising: Too early to tell
Valuable but imperfect: Some healthy adults should consider it, but there are major drawbacks
Harmful/scandalous: Clear serious drawbacks that last beyond cessation of treatment
Wonder-drug: Most adults should consider it
30
25
22
13
10
OptionVotes
NO
YES
1161
943
OptionVotes
NO
YES
100
100