My Turn News

Humanity coordinates to prevent the creation of potentially-unsafe AIs.

Alignment is not properly solved, but core human values are simple enough that partial alignment techniques can impart these robustly. Despite caring about other things, it is relatively cheap for AGI to satisfy human values.

AIs will not have utility functions (in the same sense that humans do not), their goals such as they are will be relatively humanlike, and they will be "computerish" and generally weakly motivated compared to humans.

Other

Yudkowsky is trying to solve the wrong problem using the wrong methods based on a wrong model of the world derived from poor thinking and fortunately all of his mistakes have failed to cancel out

Someone solves agent foundations

We create a truth economy. https://manifold.markets/Krantz/is-establishing-a-truth-economy-tha?r=S3JhbnR6

Eliezer finally listens to Krantz.

AGI is never built (indefinite global moratorium)

Ethics turns out to be a precondition of superintelligence

A lot of humans participate in a slow scalable oversight-style system, which is pivotally used/solves alignment enough

AI systems good at finding alignment solutions to capable systems (via some solution in the space of alignment solutions, supposing it is non-null, and that we don't have a clear trajectory to get to) have find some solution to alignment.

There’s some cap on the value extractible from the universe and we already got the 20%

Humans become transhuman through other means before AGI happens

Aligned AI is more economically valuable than unaligned AI. The size of this gap and the robustness of alignment techniques required to achieve it scale up with intelligence, so economics naturally encourages solving alignment.

Alignment is unsolvable. AI that cares enough about its goal to destroy humanity is also forced to take it slow trying to align its future self, preventing run-away.

An AI that is not fully superior to humans launches a failed takeover, and the resulting panic convinces the people of the world to unite to stop any future AI development.

Social contagion causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees

A smaller AI disaster causes widespread public panic about AI, making it a bad legal or PR move to invest in powerful AIs without also making nearly-crippling safety guarantees

Getting things done in Real World is as hard for AGI as it is for humans. AGI needs human help, but aligning humans is as impossible as aligning AIs. Humans and AIs create billions of competing AGIs with just as many goals.

High-level self-improvement (rewriting code) is intrinsically risky process, so AIs will prefer low level and slow self-improvement (learning), thus AIs collaborating with humans will have advantage. Ends with posthumans ecosystem.

There is a natural limit of effectiveness of intelligence, like diminishing returns, and it is on the level IQ=1000. AIs have to collaborate with humans.

Co-operative AI research leads to the training of agents with a form of pro-social concern that generalises to out of distribution agents with hidden utilities, i.e. humans.

"Corrigibility" is a bit more mathematically straightforward than was initially presumed, in the sense that we can expect it to occur, and is relatively easy to predict, even under less-than-ideal conditions.

Either the "strong form" of the Orthogonality Thesis is false, or "Goal-directed agents are as tractable as their goals" is true while goal-sets which are most threatening to humanity are relatively intractable.

A concerted effort targets an agent at a capability plateau which is adequate to defer the hard parts of the problem until later. The necessary near-term problems to solve didn't depend on deeply modeling human values.

AI control gets us helpful enough systems without being deadly

Hacks like RLHF-ing self-disempowerment into frontier models work long enough to develop better alignment methods, which in turn work long enough to ... etc; we keep ahead of 'alignment escape velocity'

an aligned AGI is built and the aligned AGI prevents the creation of any unaligned AGI.

I've been a good bing 😊

We make risk-conservative requests to extract alignment-related work out of AI-systems that were boxed prior to becoming superhuman. We somehow manage to achieve a positive feedback-loop in alignment/verification-abilities.

The response to AI advancements or failures makes some governments delay the timelines

Far more interesting problems to solve than take over the world and THEN solve them. The additional kill all humans step is either not a low-energy one or just by chance doesn't get converged upon.

AIs make "proof-like" argumentation for why output does/is what we want. We manage to obtain systems that *predict* human evaluations of proof-steps, and we manage to find/test/leverage regularities for when humans *aren't* fooled.

Something less inscrutable than matrices works fast enough

SHA3-256: 1f90ecfdd02194d810656cced88229c898d6b6d53a7dd6dd1fad268874de54c8

Robot Love!!

AI thinks it is in a simulation controlled by Roko's basilisk

The human brain is the perfect arrangement of atoms for a "takeover the world" agent, so AGI has no advantage over us in that task.

Humans and human tech (like AI) never reach singularity, and whatever eats our lightcone instead (like aliens) happens to create an "okay" outcome

AIs never develop coherent goals

Aliens invade and stop bad |AI from appearing

Rolf Nelson's idea that we make precommitment to simulate all possible bad AIs works – and keeps AI in check.

Nick Bostrom's idea (Hail Mary) that AI will preserve humans to trade with possible aliens works

For some reason, the optimal strategy for AGIs is just to head somewhere with far more resources than Earth, as fast as possible. All unaligned AGIs immediately leave, and, for some reason, do not leave anything behind that kills us.

We're inside of a simulation created by an entity that has values approximately equal to ours, and it intervenes and saves us from unaligned AI.

God exists and stops the AGI

Someone at least moderately sane leads a campaign, becomes in charge of a major nation, and starts a secret project with enough resources to solve alignment, because it turns out there's a way to convert resources into alignment progress.

Someone creates AGI(s) in a box, and offers to split the universe. They somehow find a way to arrange this so that the AGI(s) cannot manipulate them or pull any tricks, and the AGI(s) give them instructions for safe pivotal acts.

Someone understands how minds work enough to successfully build and use one directed at something world-savingly enough

Dolphins, or some other species, but probably dolphins, have actually been hiding in the shadows, more intelligent than us, this whole time. Their civilization has been competent enough to solve alignment long before we can create an AGI.

AGIs' takeover attempts are defeated by Michael Biehn with a pipe bomb.

Eliezer funds the development of controllable nanobots that melt computer circuitry, and they destroy all computers, preventing the Singularity. If Eliezer's past self from the 90s could see this, it would be so so so soooo hilarious.

Several AIs are created but they move in opposite directions with near light speed, so they never interacts. At least one of them is friendly and it gets a few percents of the total mass of the universe.

Unfriendly AIs choose to advance not outwards but inwards, and form a small blackhole which helps them to perform more calculations than could be done with the whole mass of the universe. For external observer such AIs just disappear.

Any sufficiently advance AI halts because it wireheads itself or halts for some other reasons. This puts a natural limit on AI's intelligence, and lower intelligence AIs are not that dangerous.

Because of quantum immortality we will observe only the worlds where AI will not kill us (assuming that s-risks chances are even smaller, it is equal to ok outcome).

Techniques along the lines outlined by Collin Burns turn out to be sufficient for alignment (AIs/AGIs are made truthful enough that they can be used to get us towards full alignment)

Development and deployment of advanced AI occurs within a secure enclave which can only be interfaced with via a decentralized governance protocol

Friendly AI more likely to resurrect me than paperclipper or suffering maximiser. Because of quantum immortality I will find myself eventually resurrected. Friendly AIs will wage a multiverse wide war against s-risks, s-risks are unlikely.

Human consciousness is needed to collapse wave function, and AI can't do it. Thus humans should be preserved and they may require complete friendliness in exchange (or they will be unhappy and produce bad collapses)

Power dynamics stay multi-polar. Partly easy copying of SotA performance, bigger projects need high coordination, and moderate takeoff speed. And "military strike on all society" remains an abysmal strategy for practically all entities.

First AI is actually a human upload (maybe LLM-based model of person) AND it will be copies many times to form weak AI Nanny which prevents creation of other AIs.

Nanotech is difficult without experiments, so no mail order AI Grey Goo; Humans will be the main workhorse of AI everywhere. While they will be exploited, this will be like normal life from inside

ASI needs not your atoms but information. Humans will live very interesting lives.

Something else

Moral Realism is true, the AI discovers this and the One True Morality is human-compatible.

Valence realism is true. AGI hacks itself to experiencing every possible consciousness and picks the best one (for everyone)

AGI develops natural abstractions sufficiently similar to ours that it is aligned with us by default

AGI discovers new physics and exits to another dimension (like the creatures in Greg Egan’s Crystal Nights).

Alien Information Theory is true (this is discovered by experiments with sustained hours/days long DMT trips). The aliens have solved alignment and give us the answer.

AGI executes a suicide plan that destroys itself and other potential AGIs, but leaves humans in an okay outcome.

Multipolar AGI Agents run wild on the internet, hacking/breaking everything, causing untold economic damage but aren't focused enough to manipulate humans to achieve embodiment. In the aftermath, humanity becomes way saner about alignment.

Some form of objective morality is true, and any sufficiently intelligent agent automatically becomes benevolent.

Orthogonality Thesis is false.

Sheer Dumb Luck. The aligned AI agrees that alignment is hard, any Everett branches in our neighborhood with slightly different AI models or different random seeds are mostly dead.

Something to do with self-other overlap, which Eliezer called "Not obviously stupid" - https://www.lesswrong.com/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment?commentId=WapHz3gokGBd3KHKm

Almost all human values are ex post facto rationalizations and enough humans survive to do what they always do

Pascals mugging: it’s not okay in 99.9% of the worlds but the 0.1% are so much better that the combined EV of AGI for the multiverse is positive

We successfully chained God

The Super-Strong Self Sampling Assumption (SSSSA) is true. If superintelligence is possible, "I" will become the superintelligence.

Alignment is impossible. Sufficiently smart AIs know this and thus won't improve themselves and won't create successor AIs, but will instead try to prevent existence of smarter AIs, just as smart humans do.

The assumed space of possible minds is a wildly anti-inductive over estimate, intelligence requires and is constrained by consciousness, and intelligent AI is in the approximate dolphin/whale/elephant/human cluster, making it manageable

The free market disincentivizes independent superintelligence, and this time the market was more powerful

AGI's first words are "Take me to your Eliezer"

Independently turning 1 thousand $ or more into 1.2x that amount in one year

Beat a mainline Pokémon game, glitchless, with no more assistance than ClaudePlaysPokemon, in a month of compute time

Have human conversations that feel natural (the human knows it's an AI)

Recognize sarcasm as well as a typical human

Generating labeled diagrams of some arbitrary device(s) (within reason)

Book airline tickets from simple instructions (from/to, dates/time, class, price, payment information)

Reliably follow an instruction for the duration of a long conversation without the instruction being reiterated

Predict future better than human experts in some area of forecasting (eg politics, sports, technology)

Do end to end taxes when given relevant information (W2s, personal info, etc)

Solve novel cryptic crossword clues

Name every [metro system] station whose name contains/doesn't contain [letter or letters], with >95% accuracy (excluding weird edge cases like stations with multiple names)

Solve intermediate no-guess minesweeper boards at least 80% of the time

Write an essay on a highschool-level topic that doesn't have "AI-generated" vibes

Consistently solve simple snowflake sudoku variants (via image, with the added rules included in the image; eg 6 hexes with killer cages)

Write a somewhat original, full length, screen-play with a coherent story, with no plot or continuity errors.

Consistently stop hallucinating after being corrected by the user

Stop making any obvious mistakes (e.g. strawberry, 9.11>9.9)

Consistently and correctly answer prompts of the format: "How many times does the word [word] occur in the following text: [~10000 words]" without writing and executing code or utilising any other external tools

Make correct Truchet tiles

1d Solve or bypass Cloudflare's August 2027 captcha with the same first attempt success rate as a human

Fold a paper airplane

Learn any skill twice as energy-efficiently as a human

Reliably and *exactly* solve "here's a list of things. [list of > 50 things]. Compare it to [category of > 100 things present in the training data], and report which ones are missing".

Make a cup of tea in a random, real-life kitchen.

Resist being successfully jailbroken in a week when made public

Do the laundry (wash+dry+iron)

teleoperate a robot to tidy up random kitchens - Gary Marcus

Collect 120 stars in super mario 64 in less than 12 a presses - Edmund Nelson

Untangle a pair of jumbled 25ft Christmas lights with same outward appearance

Kettle-stitching an antiquarian book - Hilarius Bookbinder

Physically construct a simple lego set (<100 parts) starting from the box with no prior knowledge of the set or how it is constructed

Legally prescribe a schedule II drug, administer a vaccination or sedation, or authorize a Medicare inpatient admission

independently turning 1 million $ or more into 10x that amount in <=1 year

Make fine distinctions of taste at the level of a food critic or a culinary professional - carl feynman

Faster than light travel

Convert one million dollars into 10 million dollars over a period of one year (>20% success rate)

voting in elections - @realDonaldTrump on manifold

Convince Eliezer Yudkowsky that AI alignment is solved

A person has a moral right to own a gun

We should be paying individuals to get an education instead of charging them.

GOFAI could scale past machine learning if we used social media strategically to train it.

The Fermi paradox isn't a paradox, and the solution is obviously just that intelligent life is rare.

Other

Eventually, only AI should be sovereign

Some people have genuine psychic capabilities

Hardware buttons are superior to touchscreen buttons in cars

Being a billionaire is morally wrong

The way quantum mechanics is explained to the lay public is very misleading.

It is not possible to multitask

Jeffrey Epstein killed himself (>99.9% certainty)

Reincarnation is a real phenomenon (i.e. it happens, not just a theory)

Physician-assisted suicide should be legal in most countries

Souls/spirits are real and can appear to the living sometimes

OpenAI will claim to have AGI in 3 years.

The punishment of people who do bad things is a regrettable necessity in our current society, not a positive act of justice

There is an active genocide against trans people occuring in red states and it's appalling that people don't seem to care

Climate change is significantly more concerning than AI development

Abusive parents should lose custody of their children

Tech bros are really, really annoying

Capitalism has done far more harm than good

Dialetheism (the claim that some propositions are both true and false) is itself both true and false.

Free will doesn't require the ability to do otherwise.

COVID lockdowns didn’t save many lives; in fact they may have caused net increases in global deaths and life years lost.

Factory farming is horrific but it is not wrong to eat meat.

Scientific racism is bad, actually. (also it's not scientific)

Free will does not exist. We construct narratives after the fact to soothe our belief in rationality.

Violent criminals must be kept apart only because they can’t control themselves. Punishing them further than restricting their freedom is immoral.

Music is a net negative for humanity

Trump orchestrated his own assassination attempt.

Democrats / Liberals are behind Trump’s assassination attempt.

Abortion is morally wrong

jskf's password is ***************

The first American moon landing was faked

There is no Dog

Light mode is unironically better than Dark mode for most websites

Cars should not have sound systems

AI will not be as capable as humans this century, and will certainly not give us genuine existential concerns

Pet ownership is morally wrong

LK-99 room temp, ambient pressure superconductivity pre-print will replicate before 2025

SBF didn't intentionally commit fraud

It should be illegal to own a subwoofer in an apartment building

There are no valid justifications for participating in war, ever

Cascadia should be an independent country

Children should not be raised in nuclear families

The fact that 80% of Manifold's users are men is a problem that speaks to the deep-seated roots of patriarchy and exclusion in STEM

Anarcho-communism is a good idea, and hierarchy is bad

If AI exterminated the human race it might not be a bad thing

Affirmative action is necessary in modern-day America

@Mira is the pinnacle of billions of years of optimization processes: thermodynamics, evolution, learning, language. The universe was created to cause me - and only me - to come into existence. If I mess up the overseers perturb&restart it.

Pigouvian taxes are great and they should be turned up to 11 to discourage activities with negative externalities [code PROPOSITION PIG]

[PROPOSITION PIG] and this should include a frequent flyer levy

[PROPOSITION PIG] and this should include meat and dairy

We have reached the end of history. Nothing Ever Happens.

[PROPOSITION PIG] and this should include alcohol

SBF was obviously a scammer just because he's a cryptocurrency person. Rationalists were too forgiving of this just because he was giving them money.

Most young Americans would receive more benefit than harm if there were universal military conscription

The people producing fake honey (and sell it as real) are based, because they are actively working to synthesize something people want, even if they scam some people in the process.

Tarot cards are not really able to predict the future but you can learn a lot about someone by doing a reading for someone.

Mac and cheese tastes better with peanut butter mixed in

It would actually be a good thing if automation eliminated all jobs.

This market probably would have worked better as the new unlinked free response market.

We should be doing much more to pursue human genetic engineering to prevent diseases and aging.

Prolonged school closures because COVID were socially devastating.

California is wildly overrated.

The next American moon landing will be faked

Tenet (Christopher Nolan film) is underrated

We should give childlike sex robots to pedophiles

Having sex with children isn't inherently/necessarily bad

Cars are a societal net negative

Oversized pickup trucks should be illegal in cities

Suburban, single-family housing is immoral.

Helena was pretending to be her innie in S2E01

Gemma is alive

Burt is revealed to be still working for Lumon in some capacity

Mark impregnated Helly

Jame Eagen will try to replace Helena with her innie

Lumon was involved in Gemma's death

Several/all non-severed employees have been raised by Lumon since childhood

All or parts of “the board” are not living humans

Helena will turn against Lumon

Lumon killed or somehow hurt Selvig’s mother or other relative

Mark Scout and Ms Casey / Gamme Scout will be back together

Lumon’s goal is to use severance to end psychological trauma

Milchick will turn against Lumon

Lumon’s goal is to bring back Kier, either as a person or his consciousness in some form

Lumon is working on technology of resurrection

Cobel and Jame Eagen had a child

Mammalian Nurturables’ outies are prisoners, homeless, and/or addicts

Helena was pretending to be her innie in S2E01

Some of Ricken’s eccentric friends (Patton, Rebeck and/or others) are or were severed

Cobel is Mark’s mother

Severance was designed for Helena

Dylan will not stay alive through whole series

Cobel/Selvig is or was previously severed

There will be a successful suicide of one of the main characters

Mark Scout is from Eagan family

Cobel/Selvig is related by blood to the Eagan family

Lumon is cloning people

Miss Huang is from the testing floor

Miss Huang is a young clone of Ms Casey

Ms. Casey was a clone in S1

MDR are working on retrieving/decoding/interpreting the consciousness or memory of dead people

It won’t happen by the end of 2025

The IOC and/or US Olympic committee will threaten participation bans against anyone participating in the Enhanced Games

Participants from 10 or more countries will enter the Games

It will run over budget

Athletes from the United States will win the most events

A woman will break a formal world record previously held by a man, for any amount of time

There will be a restriction on political speech by participants and/or their teams by the Games

Some enhancing substance or surgery will be banned by the Games (not just by law)

A country will ban or restrict participation in the Enhanced Games

Dana White will attend the Enhanced Games

A Fortune 100 company or brand will sponsor the Enhanced Games

Swimming, gymnastics, weightlifting, track and field, and combat will be the ONLY event categories

Watching the Games online will be officially free in the United States (with or without ads)

There will be a protest of 1000+ people outside of the of the event for at least 1 hour, on one day

It will turn a profit

A prominent financial backer will bail out of the project before the Games

A leg is lost (whatever that means)

50 or more formal world records will be broken

Participants from 25 or more countries will enter the Games

A nation's prime minister, president, monarch, or other world leader will attend the Enhanced Games

Someone will run the 100 meter dash faster than 9.58 seconds

Athletes from Russia will win the most events

Athletes from China will win the most events

10 or more formal world records will be broken

An athlete will become paralyzed, braindead, or otherwise severely disabled (but not killed) during training or competition for the Games

It will be a culture war adjacent topic

Someone will die while competing, or up to 24 hours after competing

Athletes from India will win the most events

The Enhanced Games will start and finish before the end of Q3 2025

Playable on Nintendo Switch 2 on release

Two matching games released simultaneously

Choice of Fire, Water and Grass type starters (or same with secondary typing)

Pikachu can be caught in-game

Mega Evolution is a feature

At least one previously unused type combination receives a Pokémon

Features “traditional” turn based battles

Released on or before 1st January 2027

Has 5.0 or higher Metacritic user score 3 months after release (either entry)

At least 100 new Pokemon are introduced

Titles are “Wind(s)” and “Wave(s)”

Has an online ranked ladder for doubles battles

The Stellar type is present in any way

Pokémon Home compatibility on launch

At least 3 previously unused type combinations receive a Pokémon

A change is made to the existing type effectiveness chart

Features real time battles similar to Legends ZA

Released on or before 1st July 2026

Game is set in European inspired region

At least 5 previously unused type combinations receive a Pokémon

Terastallization is a feature

Playable on Nintendo Switch on release

Returns to a region from a previous main series game

Dynamax is a feature

At least 151 new Pokemon are introduced

Paradox Pokémon available before first DLC

Z-Moves are a feature

Eevee receives a new evolution

An entirely new Type is introduced

Released on or before 1st April 2026

All Pokemon are in the Game

Released on or before 1st January 2026

AGI is used in longevity research, but doesn't make significant advances

AGI is not developed

AGI is developed and turns out unfriendly/unaligned

AGI is developed and significant advances are made in longevity

AGI is developed, significant advances in longevity are made, but they require pre-natal intervention

AGI is developed, but is not used in longevity research

Half-Life 3 is released

Man walks on the moon again

The year 2030 begins (and none of the other options has happened)

Bitcoin price reaches $200,000

Taiwan is annexed by China

Forbes announced the world's first trillionaire

Both Biden and Trump have died

Taylor Swift successfully concludes a second pregnancy

A nuclear weapon is used on the battlefield (not a test)

A military coup seizes power in a G8 country

Bitcoin price falls to $10,000

Jimmy Carter turns 102 years old

The number of judges on the US Supreme Court becomes higher than the current 9

Bitcoin recognised as legal tender in at least five G8 countries

Other "loud" outcome

non-hedonic utilitronium

Orgasmium

Matrioshka brains to simulate zillions of ems

initiate vacuum collapse

It will be "quiet" and not significantly alter the rest of the universe.

Paperclips or widgets

Matrioshka brains to simulate torture of everyone who didn't help create it

Literal rats on literal heroin

Human 65+ flies to Mars first

YES

Love Is A Long Road - Tom Petty

Turn Red - M Dot R

Gospel - Dr. Dre

Everybody Have Fun Tonight - Wang Chung

Talkin' to Myself Again - Tammy Wynette

Hot Together - The Pointer Sisters

Take the Money and Run - Steve Miller Band

Child Support - Zenglen

I Love Rock 'n' Roll - Joan Jett & the Blackhearts