Pascal News

Other

Humanity coordinates to prevent the creation of potentially-unsafe AIs.

AIs will not have utility functions (in the same sense that humans do not), their goals such as they are will be relatively humanlike, and they will be "computerish" and generally weakly motivated compared to humans.

Alignment is not properly solved, but core human values are simple enough that partial alignment techniques can impart these robustly. Despite caring about other things, it is relatively cheap for AGI to satisfy human values.

Yudkowsky is trying to solve the wrong problem using the wrong methods based on a wrong model of the world derived from poor thinking and fortunately all of his mistakes have failed to cancel out

AGI is never built (indefinite global moratorium)

Eliezer finally listens to Krantz [resolves NO]

We make risk-conservative requests to extract alignment-related work out of AI-systems that were boxed prior to becoming superhuman. We somehow manage to achieve a positive feedback-loop in alignment/verification-abilities.

Someone solves agent foundations

Valence realism is true. AGI hacks itself to experiencing every possible consciousness and picks the best one (for everyone)

Multipolar AGI Agents run wild on the internet, hacking/breaking everything, causing untold economic damage but aren't focused enough to manipulate humans to achieve embodiment. In the aftermath, humanity becomes way saner about alignment.

We create a truth economy. https://manifold.markets/Krantz/is-establishing-a-truth-economy-tha?r=S3JhbnR6

Either the "strong form" of the Orthogonality Thesis is false, or "Goal-directed agents are as tractable as their goals" is true while goal-sets which are most threatening to humanity are relatively intractable.

Ethics turns out to be a precondition of superintelligence

AIs make "proof-like" argumentation for why output does/is what we want. We manage to obtain systems that *predict* human evaluations of proof-steps, and we manage to find/test/leverage regularities for when humans *aren't* fooled.

A lot of humans participate in a slow scalable oversight-style system, which is pivotally used/solves alignment enough

Humans become transhuman through other means before AGI happens

Humans and human tech (like AI) never reach singularity, and whatever eats our lightcone instead (like aliens) happens to create an "okay" outcome

AIs never develop coherent goals

Nick Bostrom's idea (Hail Mary) that AI will preserve humans to trade with possible aliens works

An AI that is not fully superior to humans launches a failed takeover, and the resulting panic convinces the people of the world to unite to stop any future AI development.

Someone creates AGI(s) in a box, and offers to split the universe. They somehow find a way to arrange this so that the AGI(s) cannot manipulate them or pull any tricks, and the AGI(s) give them instructions for safe pivotal acts.

Getting things done in Real World is as hard for AGI as it is for humans. AGI needs human help, but aligning humans is as impossible as aligning AIs. Humans and AIs create billions of competing AGIs with just as many goals.

Development and deployment of advanced AI occurs within a secure enclave which can only be interfaced with via a decentralized governance protocol

High-level self-improvement (rewriting code) is intrinsically risky process, so AIs will prefer low level and slow self-improvement (learning), thus AIs collaborating with humans will have advantage. Ends with posthumans ecosystem.

Human consciousness is needed to collapse wave function, and AI can't do it. Thus humans should be preserved and they may require complete friendliness in exchange (or they will be unhappy and produce bad collapses)

Power dynamics stay multi-polar. Partly easy copying of SotA performance, bigger projects need high coordination, and moderate takeoff speed. And "military strike on all society" remains an abysmal strategy for practically all entities.

ASI needs not your atoms but information. Humans will live very interesting lives.

Something else

Moral Realism is true, the AI discovers this and the One True Morality is human-compatible.

AGI develops natural abstractions sufficiently similar to ours that it is aligned with us by default

Co-operative AI research leads to the training of agents with a form of pro-social concern that generalises to out of distribution agents with hidden utilities, i.e. humans.

Orthogonality Thesis is false.

"Corrigibility" is a bit more mathematically straightforward than was initially presumed, in the sense that we can expect it to occur, and is relatively easy to predict, even under less-than-ideal conditions.

A concerted effort targets an agent at a capability plateau which is adequate to defer the hard parts of the problem until later. The necessary near-term problems to solve didn't depend on deeply modeling human values.

AI control gets us helpful enough systems without being deadly

Alignment is impossible. Sufficiently smart AIs know this and thus won't improve themselves and won't create successor AIs, but will instead try to prevent existence of smarter AIs, just as smart humans do.

Hacks like RLHF-ing self-disempowerment into frontier models work long enough to develop better alignment methods, which in turn work long enough to ... etc; we keep ahead of 'alignment escape velocity'

an aligned AGI is built and the aligned AGI prevents the creation of any unaligned AGI.

I've been a good bing 😊

The response to AI advancements or failures makes some governments delay the timelines

Far more interesting problems to solve than take over the world and THEN solve them. The additional kill all humans step is either not a low-energy one or just by chance doesn't get converged upon.

AI systems good at finding alignment solutions to capable systems (via some solution in the space of alignment solutions, supposing it is non-null, and that we don't have a clear trajectory to get to) have find some solution to alignment.

Something less inscrutable than matrices works fast enough

There’s some cap on the value extractible from the universe and we already got the 20%

SHA3-256: 1f90ecfdd02194d810656cced88229c898d6b6d53a7dd6dd1fad268874de54c8

Robot Love!!

AI thinks it is in a simulation controlled by Roko's basilisk

The human brain is the perfect arrangement of atoms for a "takeover the world" agent, so AGI has no advantage over us in that task.

Aligned AI is more economically valuable than unaligned AI. The size of this gap and the robustness of alignment techniques required to achieve it scale up with intelligence, so economics naturally encourages solving alignment.

Alignment is unsolvable. AI that cares enough about its goal to destroy humanity is also forced to take it slow trying to align its future self, preventing run-away.

Aliens invade and stop bad |AI from appearing

Rolf Nelson's idea that we make precommitment to simulate all possible bad AIs works – and keeps AI in check.

For some reason, the optimal strategy for AGIs is just to head somewhere with far more resources than Earth, as fast as possible. All unaligned AGIs immediately leave, and, for some reason, do not leave anything behind that kills us.

We're inside of a simulation created by an entity that has values approximately equal to ours, and it intervenes and saves us from unaligned AI.

God exists and stops the AGI

Someone at least moderately sane leads a campaign, becomes in charge of a major nation, and starts a secret project with enough resources to solve alignment, because it turns out there's a way to convert resources into alignment progress.

Someone understands how minds work enough to successfully build and use one directed at something world-savingly enough

Dolphins, or some other species, but probably dolphins, have actually been hiding in the shadows, more intelligent than us, this whole time. Their civilization has been competent enough to solve alignment long before we can create an AGI.

AGIs' takeover attempts are defeated by Michael Biehn with a pipe bomb.

Eliezer funds the development of controllable nanobots that melt computer circuitry, and they destroy all computers, preventing the Singularity. If Eliezer's past self from the 90s could see this, it would be so so so soooo hilarious.

Several AIs are created but they move in opposite directions with near light speed, so they never interacts. At least one of them is friendly and it gets a few percents of the total mass of the universe.

Unfriendly AIs choose to advance not outwards but inwards, and form a small blackhole which helps them to perform more calculations than could be done with the whole mass of the universe. For external observer such AIs just disappear.

Any sufficiently advance AI halts because it wireheads itself or halts for some other reasons. This puts a natural limit on AI's intelligence, and lower intelligence AIs are not that dangerous.

Because of quantum immortality we will observe only the worlds where AI will not kill us (assuming that s-risks chances are even smaller, it is equal to ok outcome).

Techniques along the lines outlined by Collin Burns turn out to be sufficient for alignment (AIs/AGIs are made truthful enough that they can be used to get us towards full alignment)

Social contagion causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees

A smaller AI disaster causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees

Friendly AI more likely to resurrect me than paperclipper or suffering maximiser. Because of quantum immortality I will find myself eventually resurrected. Friendly AIs will wage a multiverse wide war against s-risks, s-risks are unlikely.

First AI is actually a human upload (maybe LLM-based model of person) AND it will be copies many times to form weak AI Nanny which prevents creation of other AIs.

There is a natural limit of effectiveness of intelligence, like diminishing returns, and it is on the level IQ=1000. AIs have to collaborate with humans.

Nanotech is difficult without experiments, so no mail order AI Grey Goo; Humans will be the main workhorse of AI everywhere. While they will be exploited, this will be like normal life from inside

AGI discovers new physics and exits to another dimension (like the creatures in Greg Egan’s Crystal Nights).

Alien Information Theory is true (this is discovered by experiments with sustained hours/days long DMT trips). The aliens have solved alignment and give us the answer.

AGI executes a suicide plan that destroys itself and other potential AGIs, but leaves humans in an okay outcome.

Some form of objective morality is true, and any sufficiently intelligent agent automatically becomes benevolent.

Sheer Dumb Luck. The aligned AI agrees that alignment is hard, any Everett branches in our neighborhood with slightly different AI models or different random seeds are mostly dead.

Something to do with self-other overlap, which Eliezer called "Not obviously stupid" - https://www.lesswrong.com/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment?commentId=WapHz3gokGBd3KHKm

Almost all human values are ex post facto rationalizations and enough humans survive to do what they always do

Pascals mugging: it’s not okay in 99.9% of the worlds but the 0.1% are so much better that the combined EV of AGI for the multiverse is positive

We successfully chained God

The Super-Strong Self Sampling Assumption (SSSSA) is true. If superintelligence is possible, "I" will become the superintelligence.

The assumed space of possible minds is a wildly anti-inductive over estimate, intelligence requires and is constrained by consciousness, and intelligent AI is in the approximate dolphin/whale/elephant/human cluster, making it manageable

The free market disincentivizes independent superintelligence, and this time the market was more powerful

AGI's first words are "Take me to your Eliezer"

Callum Turner

Josh O'Connor

Other

Jacob Elordi

Aaron Taylor-Johnson

Jack Lowden

Anthony Boyle

Pedro Pascal

Tom Hardy

Idris Elba

Taron Egerton

Cillian Murphy

Tom Hiddleston

Regé-Jean Page

Lashana Lynch

Henry Cavill

Sam Heughan

Dev Patel

Robert Pattinson

Theo James

Damson Idris

James Norton

Daryl McCormack

Paapa Essiedu

Tom Cruise

Brad Pitt

George Clooney

Matt Damon

Kingsley Ben-Adir

Richard Madden

Paul Mescal

Sam Claflin

John Boyega

Pierce Brosnan

Sope Dirisu

Chiwetel Ejiofor

Robert Aramayo

Liam Hemsworth

Andrew Garfield

Will Poulter

Nicholas Hoult

Miles Teller

Aidan Turner

Tom Hopper

Lucien Laviscount

Harry Styles

Luke Evans

Timothé Chalamet

Tom Holland

Harris Dickinson

Eliezer Yudkowsky

Emma Watson

Jamal Musiala

Florian Wirtz

Kai Havertz

Joshua Kimmich

Jonathan Tah

Oliver Baumann

Nico Schlotterbeck

David Raum

Pascal Groß

Aleksandar Pavlovic

Alexander Nübel

Nick Woltemade

Serge Gnabry

Antonio Rüdiger

Leon Goretzka

Leroy Sané

Jonas Urbig

Felix Nmecha

Malick Thiaw

Lennart Karl

Niclas Füllkrug

Jamie Leweling

Deniz Undav

Nathaniel Brown

Ridle Baku

Karim Adeyemi

Josha Vagnoman

Anton Stach

Robert Andrich

Kevin Schade

Noah Atubolu

Angelo Stiller

Jonathan Burkardt

Maximilian Mittelstädt

Anyone not on this list

Manuel Neuer

Nadiem Amiri

Paul Nebel

Robin Gosens

Finn Dahmen

Saïd El Mala

Maximilian Beier

Chris Führich

Yann Bisseck

Armel Bella-Kotchap

Tim Kleindienst

Brajan Gruda

Marc-André ter Stegen

Julian Brandt

Emre Can

Assan Ouédraogo

Tom Bischof

Philipp Treu

Thilo Kehrer

Nicolò Tresoldi

Rocco Reitz

Paul Wanner

Stefan Ortega

Matthias Ginter

Eric Martel

Luca Netz

Moritz Nicolas

Mario Götze

Fabian Reese

Nicolas Kühn

Robin Zentner

Max Finkgräfe

Niklas Beste

Marius Wolf

Jonas Hofmann

Bright Arrey-Mbi

Marvin Ducksch

Nelson Weiper

Paris Brunner

Finn Jeltsch

Diant Ramaj

Janis Blaswich

Kevin Behrens

Grischa Prömel

Youssoufa Moukoko

Marvin Schwäbe

Frans Krätzig

Kennet Eichhorn

Bence Dardai

Robert Downey, Jr. (Victor von Doom / Doctor Doom, Tony Stark / Iron Man)

Tom Holland (Peter Parker / Spider-Man)

Anthony Mackie (Sam Wilson / Captain America)

Pedro Pascal (Reed Richards / Mister Fantastic)

Benedict Cumberbatch (Dr. Stephen Strange)

Ebon Moss-Bachrach (Ben Grimm / The Thing)

Joseph Quinn (Johnny Storm / Human Torch)

Samuel L. Jackson (Nick Fury)

Sebastian Stan (Bucky Barnes / Winter Soldier / White Wolf)

Vanessa Kirby (Sue Storm / Invisible Woman)

Simu Liu (Shang-Chi)

Paul Rudd (Scott Lang / Ant-Man)

Charlie Cox (Matt Murdock / Daredevil)

Hannah John-Kamen (Ava Starr / Ghost)

Julia Louis-Dreyfus (Valentina Allegra de Fontaine)

Pom Klementieff (Mantis)

David Harbour (Alexei Shostakov / Red Guardian)

Danny Ramirez (Joaquin Torres / Falcon)

Chris Hemsworth (Thor)

Iman Vellani (Kamala Khan / Ms. Marvel)

Mark Ruffalo (Bruce Banner / Hulk)

Dominique Thorne (Riri Williams / Ironheart)

Florence Pugh (Yelena Belova / Black Widow)

Benedict Wong (Wong)

Chris Pratt (Peter Quill / Star-Lord)

Vin Diesel (Groot)

Ryan Reynolds (Wade Wilson / Deadpool)

Jeremy Renner (Clint Barton / Hawkeye)

Wyatt Russell (John Walker / U.S. Agent)

Brie Larson (Carol Danvers / Captain Marvel)

Letitia Wright (Shuri / Black Panther)

Karen Gillan (Nebula)

Tatiana Maslany (Jennifer Walters / She-Hulk)

Hailee Steinfeld (Kate Bishop / Hawkeye)

Teyonah Parris (Monica Rambeau)

Xochitl Gomez (America Chavez)

Hayley Atwell (Peggy Carter / Captain Carter)

Kathryn Newton (Cassie Lang)

Bradley Cooper (Rocket Raccoon)

Will Poulter (Adam Warlock)

Tenoch Huerta Mejía (Namor)

Don Cheadle (James "Rhodey" Rhodes / War Machine)

Tessa Thompson (Valkyrie)

Winston Duke (M'Baku)

Elizabeth Olsen (Wanda Maximoff / Scarlet Witch)

Paul Bettany (Vision)

Chris Evans (Steve Rogers / Captain America)

Kathryn Hahn (Agatha Harkness)

Danai Gurira (Okoye)

Oscar Isaac (Marc Spector / Moon Knight, Steven Grant / Mr. Knight, Jake Lockley)

Hugh Jackman (James "Logan" Howlett / Wolverine)

Tom Hiddleston (Loki)

Mahershala Ali (Blade)

Sean Gunn (Kraglin)

Alaqua Cox (Maya Lopez / Echo)

Lupita Nyong'o (Nakia)

Wil Deusner (Skaar)

Gemma Chan (Sersi)

Kumail Nanjiani (Kingo)

Lia McHugh (Sprite)

Brian Tyree Henry (Phastos)

Barry Keoghan (Druig)

Harry Styles (Eros / Starfox)

Patton Oswalt (Pip the Troll)

Kit Harington (Dane Whitman)

Yahya Abdul-Mateen II (Simon Williams / Wonder Man)

Evangeline Lilly (Hope van Dyne / Wasp)

Seth Green (Howard the Duck)

Zoe Saldaña (Gamora)

Dave Bautista (Drax)

Angelina Jolie (Thena)

Gael García Bernal (Jack Russell / Werewolf by Night)

Scarlett Johansson (Natasha Romanoff / Black Widow)

Andrew Garfield (Peter Parker / Spider-Man)

Lauren Ridloff (Makkari)

Emilia Clarke (G'iah)

Ralph Ineson (Galactus)

May Calamawy (Layla El-Faouly / Scarlet Scarab)

Jeffrey Wright (The Watcher)

Tobey Maguire (Peter Parker / Spider-Man)

Harrison Ford (Thaddeus "Thunderbolt" Ross)

Brett Goldstein (Hercules)

Don Lee (Gilgamesh)

YES

NO

YES

YES

YES

NO

Taylor Barnard

Joseph Maria Martí

Antonio Felix DaCosta

Eduardo Mortara

Nico Müller

Oliver Rowland

Pascal Vehrlein

Maximilian Günter

Mitch Evans

Lucas Di Grassi

Joel Eriksen

Sebastien Buemi

Nyck De Vries

Zane Maloney

Norman Nato

Jean-Eric Vergne

Jake Dennis

Felipe Drugovich