Eventual News

Humanity coordinates to prevent the creation of potentially-unsafe AIs.

Alignment is not properly solved, but core human values are simple enough that partial alignment techniques can impart these robustly. Despite caring about other things, it is relatively cheap for AGI to satisfy human values.

AIs will not have utility functions (in the same sense that humans do not), their goals such as they are will be relatively humanlike, and they will be "computerish" and generally weakly motivated compared to humans.

Yudkowsky is trying to solve the wrong problem using the wrong methods based on a wrong model of the world derived from poor thinking and fortunately all of his mistakes have failed to cancel out

We create a truth economy. https://manifold.markets/Krantz/is-establishing-a-truth-economy-tha?r=S3JhbnR6

Eliezer finally listens to Krantz.

Ethics turns out to be a precondition of superintelligence

Other

Someone solves agent foundations

A smaller AI disaster causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees

Something less inscrutable than matrices works fast enough

Nanotech is difficult without experiments, so no mail order AI Grey Goo; Humans will be the main workhorse of AI everywhere. While they will be exploited, this will be like normal life from inside

Orthogonality Thesis is false.

We make risk-conservative requests to extract alignment-related work out of AI-systems that were boxed prior to becoming superhuman. We somehow manage to achieve a positive feedback-loop in alignment/verification-abilities.

The response to AI advancements or failures makes some governments delay the timelines

Far more interesting problems to solve than take over the world and THEN solve them. The additional kill all humans step is either not a low-energy one or just by chance doesn't get converged upon.

AIs make "proof-like" argumentation for why output does/is what we want. We manage to obtain systems that *predict* human evaluations of proof-steps, and we manage to find/test/leverage regularities for when humans *aren't* fooled.

A lot of humans participate in a slow scalable oversight-style system, which is pivotally used/solves alignment enough

There’s some cap on the value extractible from the universe and we already got the 20%

Humans become transhuman through other means before AGI happens

Aligned AI is more economically valuable than unaligned AI. The size of this gap and the robustness of alignment techniques required to achieve it scale up with intelligence, so economics naturally encourages solving alignment.

Humans and human tech (like AI) never reach singularity, and whatever eats our lightcone instead (like aliens) happens to create an "okay" outcome

Alignment is unsolvable. AI that cares enough about its goal to destroy humanity is also forced to take it slow trying to align its future self, preventing run-away.

An AI that is not fully superior to humans launches a failed takeover, and the resulting panic convinces the people of the world to unite to stop any future AI development.

Techniques along the lines outlined by Collin Burns turn out to be sufficient for alignment (AIs/AGIs are made truthful enough that they can be used to get us towards full alignment)

Social contagion causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees

Getting things done in Real World is as hard for AGI as it is for humans. AGI needs human help, but aligning humans is as impossible as aligning AIs. Humans and AIs create billions of competing AGIs with just as many goals.

Development and deployment of advanced AI occurs within a secure enclave which can only be interfaced with via a decentralized governance protocol

High-level self-improvement (rewriting code) is intrinsically risky process, so AIs will prefer low level and slow self-improvement (learning), thus AIs collaborating with humans will have advantage. Ends with posthumans ecosystem.

AGI is never built (indefinite global moratorium)

AGI develops natural abstractions sufficiently similar to ours that it is aligned with us by default

Multipolar AGI Agents run wild on the internet, hacking/breaking everything, causing untold economic damage but aren't focused enough to manipulate humans to achieve embodiment. In the aftermath, humanity becomes way saner about alignment.

Co-operative AI research leads to the training of agents with a form of pro-social concern that generalises to out of distribution agents with hidden utilities, i.e. humans.

"Corrigibility" is a bit more mathematically straightforward than was initially presumed, in the sense that we can expect it to occur, and is relatively easy to predict, even under less-than-ideal conditions.

Either the "strong form" of the Orthogonality Thesis is false, or "Goal-directed agents are as tractable as their goals" is true while goal-sets which are most threatening to humanity are relatively intractable.

A concerted effort targets an agent at a capability plateau which is adequate to defer the hard parts of the problem until later. The necessary near-term problems to solve didn't depend on deeply modeling human values.

AI control gets us helpful enough systems without being deadly

Alignment is impossible. Sufficiently smart AIs know this and thus won't improve themselves and won't create successor AIs, but will instead try to prevent existence of smarter AIs, just as smart humans do.

Hacks like RLHF-ing self-disempowerment into frontier models work long enough to develop better alignment methods, which in turn work long enough to ... etc; we keep ahead of 'alignment escape velocity'

an aligned AGI is built and the aligned AGI prevents the creation of any unaligned AGI.

I've been a good bing 😊

AI systems good at finding alignment solutions to capable systems (via some solution in the space of alignment solutions, supposing it is non-null, and that we don't have a clear trajectory to get to) have find some solution to alignment.

SHA3-256: 1f90ecfdd02194d810656cced88229c898d6b6d53a7dd6dd1fad268874de54c8

Robot Love!!

AI thinks it is in a simulation controlled by Roko's basilisk

The human brain is the perfect arrangement of atoms for a "takeover the world" agent, so AGI has no advantage over us in that task.

AIs never develop coherent goals

Aliens invade and stop bad |AI from appearing

Rolf Nelson's idea that we make precommitment to simulate all possible bad AIs works – and keeps AI in check.

Nick Bostrom's idea (Hail Mary) that AI will preserve humans to trade with possible aliens works

For some reason, the optimal strategy for AGIs is just to head somewhere with far more resources than Earth, as fast as possible. All unaligned AGIs immediately leave, and, for some reason, do not leave anything behind that kills us.

We're inside of a simulation created by an entity that has values approximately equal to ours, and it intervenes and saves us from unaligned AI.

God exists and stops the AGI

Someone at least moderately sane leads a campaign, becomes in charge of a major nation, and starts a secret project with enough resources to solve alignment, because it turns out there's a way to convert resources into alignment progress.

Someone creates AGI(s) in a box, and offers to split the universe. They somehow find a way to arrange this so that the AGI(s) cannot manipulate them or pull any tricks, and the AGI(s) give them instructions for safe pivotal acts.

Someone understands how minds work enough to successfully build and use one directed at something world-savingly enough

Dolphins, or some other species, but probably dolphins, have actually been hiding in the shadows, more intelligent than us, this whole time. Their civilization has been competent enough to solve alignment long before we can create an AGI.

AGIs' takeover attempts are defeated by Michael Biehn with a pipe bomb.

Eliezer funds the development of controllable nanobots that melt computer circuitry, and they destroy all computers, preventing the Singularity. If Eliezer's past self from the 90s could see this, it would be so so so soooo hilarious.

Several AIs are created but they move in opposite directions with near light speed, so they never interacts. At least one of them is friendly and it gets a few percents of the total mass of the universe.

Unfriendly AIs choose to advance not outwards but inwards, and form a small blackhole which helps them to perform more calculations than could be done with the whole mass of the universe. For external observer such AIs just disappear.

Any sufficiently advance AI halts because it wireheads itself or halts for some other reasons. This puts a natural limit on AI's intelligence, and lower intelligence AIs are not that dangerous.

Because of quantum immortality we will observe only the worlds where AI will not kill us (assuming that s-risks chances are even smaller, it is equal to ok outcome).

Friendly AI more likely to resurrect me than paperclipper or suffering maximiser. Because of quantum immortality I will find myself eventually resurrected. Friendly AIs will wage a multiverse wide war against s-risks, s-risks are unlikely.

Human consciousness is needed to collapse wave function, and AI can't do it. Thus humans should be preserved and they may require complete friendliness in exchange (or they will be unhappy and produce bad collapses)

Power dynamics stay multi-polar. Partly easy copying of SotA performance, bigger projects need high coordination, and moderate takeoff speed. And "military strike on all society" remains an abysmal strategy for practically all entities.

First AI is actually a human upload (maybe LLM-based model of person) AND it will be copies many times to form weak AI Nanny which prevents creation of other AIs.

There is a natural limit of effectiveness of intelligence, like diminishing returns, and it is on the level IQ=1000. AIs have to collaborate with humans.

ASI needs not your atoms but information. Humans will live very interesting lives.

Something else

Moral Realism is true, the AI discovers this and the One True Morality is human-compatible.

Valence realism is true. AGI hacks itself to experiencing every possible consciousness and picks the best one (for everyone)

AGI discovers new physics and exits to another dimension (like the creatures in Greg Egan’s Crystal Nights).

Alien Information Theory is true (this is discovered by experiments with sustained hours/days long DMT trips). The aliens have solved alignment and give us the answer.

AGI executes a suicide plan that destroys itself and other potential AGIs, but leaves humans in an okay outcome.

Some form of objective morality is true, and any sufficiently intelligent agent automatically becomes benevolent.

Sheer Dumb Luck. The aligned AI agrees that alignment is hard, any Everett branches in our neighborhood with slightly different AI models or different random seeds are mostly dead.

Something to do with self-other overlap, which Eliezer called "Not obviously stupid" - https://www.lesswrong.com/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment?commentId=WapHz3gokGBd3KHKm

Almost all human values are ex post facto rationalizations and enough humans survive to do what they always do

Pascals mugging: it’s not okay in 99.9% of the worlds but the 0.1% are so much better that the combined EV of AGI for the multiverse is positive

We successfully chained God

The Super-Strong Self Sampling Assumption (SSSSA) is true. If superintelligence is possible, "I" will become the superintelligence.

The assumed space of possible minds is a wildly anti-inductive over estimate, intelligence requires and is constrained by consciousness, and intelligent AI is in the approximate dolphin/whale/elephant/human cluster, making it manageable

The free market disincentivizes independent superintelligence, and this time the market was more powerful

AGI's first words are "Take me to your Eliezer"

[Silicon Valley] Elon Musk posts two or more tweets that are unambiguously critical of Trump by Election Day, 2028 (Nate: 70%)

[Silicon Valley] Trump establishes a Bitcoin reserve in his first 100 days (Nate: 15%)

[Economy] Trump imposes a tariff of at least 40% on China before the 2028 elections (Nate: 30%)

[Economy] Trump imposes a tariff of at least 25% on Mexico or Canada before the 2028 elections (Nate: 60%)

[Economy] The SALT deduction is raised or uncapped before the 2026 midterms (Nate: 80%)

[Economy] Trump signs legislation or passes an executive order reducing or eliminating taxes on tips by 11/7/2028 (Nate: 65%)

[Economy] There is at least one federal government shutdown before the 2028 elections (Nate: 75%)

[Foreign Policy] Trump makes any new statement that COVID probably or definitely originated in a Chinese lab by 1/20/29 (Nate: 75%)

[Constitutionality] By 1/20/29, the NYT publishes the phrase “constitutional crisis” in reference to actions undertaken by Trump, the White House, or Republicans in the headline or lede paragraph of at least three news articles (Nate: 45%)

[Trump and 2028] Conditional on his still being president, the New York Times publishes at least three articles between the 2026 midterms and 2028 election pointing toward Trump’s physical or cognitive decline (Nate: 80%)

[Constitutionality] The Supreme Court substantially overrules Trump’s interpretation of birthright citizenship (Nate: 90%)

[Culture Wars] The federal government sues any AAU university for failing to comply with Students for Fair Admissions v. Harvard by 1/20/29 (Nate: 90%)

[Immigration] Southwestern border apprehensions are at least 50% lower in 2025-2027 than on average from 2021-2023 (Nate: 75%)

[Media] MSNBC ratings are lower in 2026 than in 2018 (Nate: 80%)

[Foreign Policy] Trump visits China (Nate: 90%)

[Baseline Politics] Democrats win the House in the 2026 midterms (Nate: 85%)

[Trump Cabinet] Two or more Cabinet members from any point in Trump’s second term publicly criticize Trump by the 2028 election (Nate: 70%)

[Economy] Total federal government tax receipts, including tariffs, are lower as a share of GDP in 2027 than in 2023 (Nate: 70%)

[Constitutionality] Trump pardons any family member, not including himself, before 1/20/29 (Nate: 90%)

[Media] Trump or the federal government sue the New York Times or the Washington Post, or any active employees thereof, at any point before the 2028 election (Nate: 55%)

[Media] Fox News ratings are lower in 2026 than in 2022 (Nate: 55%)

[Trump and 2028] Trump completes his term (Nate: 75%)

[Constitutionality] Trump pardons himself before 1/20/29 (Nate: 85%)

[Baseline Politics] Mike Johnson remains Speaker of the House through 11/3/2026 (Nate: 60%)

[Economy] The US Gini coefficient is higher in 2027 than in 2023 (Nate: 85%)

[Culture Wars] Homicide rates are lower in 2027 than in 2023 (Nate: 70%)

[Electorate Demographics] The gender gap, measured as the Republican vote share among men less the share among women, will be wider in 2028 than in 2024 (Nate: 65%)

[Silicon Valley] Trump or his successors sign any law limiting the scope of Section 230 of the Communications Decency Act by 11/7/2028, reducing liability protections for social media companies (Nate: 60%)

[Trump and 2028] Trump asserts his intention on at least two occasions to run for a third term in 2028 (Nate: 40%)

[Baseline Politics] At least one new Supreme Court vacancy in the 119th or 120th Congress (Nate: 85%)

[Immigration] At least 1 million illegal/unauthorized immigrants are deported through 2027 (Nate: 90%)

[Trump and 2028] Trump’s approval rating is >=40% by the 2026 midterms (Nate: 55%)

[Baseline Politics] Democrats win the presidency in 2028 (Nate: 55%)

[Constitutionality] Trump or his successors invoke the Insurrection Act at any point before 1/20/29, even if later blocked by courts (Nate: 40%)

[Economy] The NBER declares a recession at any point before the 2028 election (Nate: 45%)

[Electorate Demographics] The Republican margin of victory or defeat improves in Santa Clara County, California in 2028 versus 2024 (Nate: 60%)

[Economy] Year-over-year inflation is >=5.0% at any point based on data published before the 2028 election (Nate: 50%)

[Constitutionality] The New York Times or Washington Post report that the Trump administration has illegally surveilled domestic political opponents by the 2028 election (a.k.a. Watergate 2.0) (Nate: 25%)

[Media] The New York Times or Washington Post hires at least one full-time explicitly pro-Trump political columnist by the 2028 election (Nate: 40%)

[Trump and 2028] Trump is impeached (again) — does not require conviction (Nate: 45%)

[Media] Trump bars all New York Times or Washington Post reporters from White House press briefings at any point before the 2028 election (Nate: 60%)

[Culture Wars] A ban on transgender student-athletes in women’s sports is passed through Congress and signed by Trump or enacted through executive action by 1/20/29 (Nate: 70%)

[Culture Wars] Childhood measles vaccination rates (currently ~93%) drop below 90% in any calendar year based on best available data as of 1/20/2029 (Nate: 35%)

[Trump and 2028] Conditional on still being president, Trump endorses the eventual winner of the 2028 GOP presidential nomination by 2/29/2028 (Nate: 60%)

[Culture Wars] Trump or any successors sign legislation with the explicit aim of increasing US fertility rates by 11/7/2028 (Nate: 40%)

[Media] Trump attends the White House Correspondents’ Association Dinner at least once (Nate: 60%)

[Trump and 2028] JD Vance is the GOP nominee in 2028 (Nate: 50%)

[Electorate Demographics] The GOP candidate increases their share of the Jewish vote in 2028 vs. Trump in 2024 (Nate: 50%)

[Foreign Policy] Trump visits Russia (Nate: 60%)

[Media] The Washington Post endorses the Democratic presidential candidate in 2028 (Nate: 60%)

[Culture Wars] The White House undertakes any executive action, or Congress passes any law, placing new restrictions on licensed online American sportsbooks by 1/20/2029 (Nate: 50%)

[Baseline Politics] Democrats win a trifecta (House + Senate + POTUS) in 2028 (Nate: 40%)

[Economy] Trump invokes the Taft-Hartley Act at least once before the 2028 elections (Nate: 25%)

[Constitutionality] Republicans hold Congressional hearings on efforts to cover up Biden’s age or physical or cognitive condition (Nate: 45%)

[Economy] The S&P 500 closes at least 40% higher on 9/1/2028 than on 1/17/2025 (5,996.66 points) (Nate: 50%)

[Constitutionality] The Economist Intelligence Unit’s Democracy Index for the US (currently 7.8) falls under 7.0 at any point before 1/20/29 (Nate: 15%)

[Economy] Trump imposes a tariff of at least 20% on Denmark before the 2028 elections (Nate: 40%)

[Silicon Valley] The FTC or another federal agency initiate a new prosecution against a Big 5 tech company by 1/20/29 (Nate: 60%)

[Constitutionality] The White House refuses to comply with any Supreme Court order (Nate: 10%)

[Culture Wars] Trump signs a bill or executive order restricting early or mail-in voting or voter registration practices for federal elections (Nate: 25%)

[Culture Wars] The Oscars rescind their DEI requirements by 2028 (Nate: 45%)

[Silicon Valley] Trump or the White House make a public statement warning of the dangers of unaligned AI by 11/7/2028 (Nate: 40%)

[Economy] Federal spending as a share of GDP is lower in 2027 than in 2023 (Nate: 60%)

[Silicon Valley] Elon Musk donates at least $100M to the Republican general election nominee in 2028 (Nate: 55%)

[Economy] Real GDP growth through Q3 2028 exceeds the average of GDP growth (3.3%) during Biden’s first 15 quarters (Nate: 35%)

[Culture Wars] At least one Oscar, Grammy, or Emmy winner explicitly praises Trump during their acceptance speech before the 2028 elections (Nate: 40%)

[Trump and 2028] Conditional on still being president, Trump endorses JD Vance by 2/29/2028 (Nate: 55%)

[Trump Cabinet] At least three Attorneys General, excluding acting AGs, occupy the position before 1/20/2029 (Nate: 25%)

[Economy] The top federal tax bracket is lower in 2028 than in 2024 (Nate: 25%)

[Electorate Demographics] The GOP candidate increases their share of the Black vote in 2028 vs. Trump in 2024 (Nate: 55%)

[Electorate Demographics] The GOP candidate increases their share of the LGBTQ vote in 2028 vs. Trump in 2024 (Nate: 55%)

[Immigration] At least 3 million illegal/unauthorized immigrants are deported through 2027 (Nate: 40%)

[Culture Wars] At least half the members of any Super Bowl champion refuse to visit the White House while Trump is in office or refuses to invite them (Nate: 10%)

[Trump and 2028] Trump’s approval rating is >=40% by the 2028 general election (Nate: 60%)

[Culture Wars] The White House undertakes any executive order, or Congress passes any law, placing new restrictions on online pornography by 1/20/2029 (Nate: 20%)

[Economy] Trump's first nominee for Federal Reserve Chair in 2026 is confirmed with at least two-thirds of voting Senators (Nate: 55%)

[Foreign Policy] Congress declares war or authorizes the use of military force by 11/7/2028 (Nate: 25%)

[Electorate Demographics] The Republican margin of victory or defeat improves in New Jersey in 2028 versus 2024 (Nate: 25%)

[Foreign Policy] Conditional on the war not ending, the US cuts foreign aid to Ukraine by at least 25% in its next budget (Nate: 10%)

[Foreign Policy] Trump recognizes Taiwan by 1/20/29 (Nate: 20%)

[Silicon Valley] At least 30% of donations from Big 5 tech employees (Amazon, Alphabet, Apple, Meta, Microsoft) go toward Republicans in the 2028 campaign (Nate: 25%)

[Silicon Valley] Any “Twitter-type” platform has more active daily users than Twitter/X by 6/30/28 (Nate: 30%)

[Electorate Demographics] The GOP candidate increases their share of the Hispanic vote in 2028 vs. Trump in 2024, as measured based on an average of the AP and Edison exit polls (Nate: 35%)

[Silicon Valley] Congress passes a shift to year-round Standard Time (Nate: 15%)

[Economy] Trump attempts to fire Jerome Powell before his term expires in May 2026 (Nate: 5%)

[Constitutionality] The Trump administration prosecutes Joe Biden (Nate: 50%)

[Baseline Politics] Democrats win the Senate in 2026 (Nate: 15%)

[Foreign Policy] There is a territorial dispute, resulting in maneuvering of military assets, between the US and any country in the Americas (including Greenland) before 1/20/29 (Nate: 20%)

[Immigration] More H1-B visas are issued on average in FY 2025-2028 versus 2021-2024 (Nate: 35%)

[Electorate Demographics] The GOP candidate increases their share of the Muslim vote in 2028 vs. Trump in 2024 (Nate: 40%)

[Culture Wars] Race relations are rated as “somewhat bad” or “very bad” by more people than in 2021 (57%) in the average of any Gallup surveys taken during Trump’s second term (Nate: 20%)

[Foreign Policy] The United States purchases or retakes any part of Greenland before 1/20/29 (Nate: 10%)

[Trump and 2028] The 2028 election is not regarded as “free or fair” (or similar terminology) by one or more major international organizations (Nate: 10%)

[Trump and 2028] The transition to the new president in 2028/2029 is marked by violence, resulting in at least five deaths (Nate: 15%)

[Culture Wars] Trump signs a bill to repeal substantial portions of Obamacare by 11/3/2026 (Nate: 30%)

[Baseline Politics] A third party is a significant factor (>=15% or more of the popular vote at any point after Labor Day) in the 2028 race (Nate: 20%)

[Trump Cabinet] At least two of Pete Hegseth, Tulsi Gabbard, and RFK Jr. do not occupy their respective cabinet positions (or are not confirmed) by the 2026 midterms (Nate: 65%)

[Foreign Policy] An American flag is planted on Mars (including by unmanned drones/robots) by 1/20/29 (Nate: 5%)

[Immigration] The border wall/fence with Mexico is completed before 1/20/29 (Nate: 20%)

[Foreign Policy] Trump or his successors successfully withdraw the US from NATO by 11/7/2028 (Nate: 10%)

[Trump and 2028] Trump’s approval rating is >=50% by the 2026 midterms (Nate: 15%)

[Trump and 2028] Trump physically refuses to leave the White House following the end of his term or removal from office (Nate: 5%)

[Media] Trump appears on Saturday Night Live at least once before the 2028 election (Nate: 30%)

[Silicon Valley] DOGE recommendations lead to cuts of at least $500 billion in government spending in any fiscal year between now and the 2028 election (Nate: 10%)

[Baseline Politics] Democrats take control of the House before the 2026 midterms through special elections or party switches (Nate: 10%)

[Foreign Policy] The United States retakes any part of the Panama Canal before 1/20/29 (Nate: 5%)

[Culture Wars] Same-sex marriage is not legal in any state as of 11/7/2028 (Nate: 5%)

[Trump and 2028] The 25th Amendment is invoked against Trump (Nate: 10%)

[Trump and 2028] A member of the Trump family (not including Trump himself) is the GOP nominee in 2028 (Nate: 5%)

[Culture Wars] Trump signs a federal abortion ban into law by 11/7/2028 (Nate: 5%)

[Trump and 2028] Trump appears on the general election ballot of enough states to win the Electoral College in 2028 (Nate: 2%)

[Trump Cabinet] Either Tulsi Gabbard or RFK Jr. are not confirmed to their current cabinet nominations (Nate: 55%)

🇺🇸 United States of America

🇷🇺 Russia

🇰🇵 North Korea

🇮🇷 Iran

🇮🇱 Israel

🇵🇰 Pakistan

Other

🇮🇳 India

Iran

South China Sea

Pakistan

India

United States

Russia

Ukraine

Taiwan

China

Venezuela

Washington DC

Guam

Belarus

Jerusalem

Okinawa

The Philippines

South Korea

Seoul

YES

NO

NO

NO

NO

Jeffrey Sachs

Huey Li (and the news)

SerpentZA

Laowhy86

Nicholas Kristof

Michael Pettis

Peter Hessler

James Fallows

Kevin D Walmsley of Inside China Business

Liziqi

Nathan Rich

Living in China

Jerry Kowal

Significant effort (10k+ posts) to manipulate comment threads on hacker news.

Asia Society Policy Institute

Yale Center Beijing

Brookings Institution (China Center)

Blondie in China

Elon Musk

NO