OptionProbability
Stop making any obvious mistakes (e.g. strawberry, 9.11>9.9)
Reliably follow an instruction for the duration of a long conversation without the instruction being reiterated
Write an essay on a highschool-level topic that doesn't have "AI-generated" vibes
Solve intermediate no-guess minesweeper boards at least 80% of the time
Generating labeled diagrams of some arbitrary device(s) (within reason)
Have human conversations that feel natural (the human knows it's an AI)
Consistently stop hallucinating after being corrected by the user
Book airline tickets from simple instructions (from/to, dates/time, class, price, payment information)
Beat a mainline Pokémon game, glitchless, with no more assistance than ClaudePlaysPokemon, in a month of compute time
Recognize sarcasm as well as a typical human
Independently turning 1 thousand $ or more into 1.2x that amount in one year
Predict future better than human experts in some area of forecasting (eg politics, sports, technology)
Consistently and correctly answer prompts of the format: "How many times does the word [word] occur in the following text: [~10000 words]" without writing and executing code or utilising any other external tools
Fold a paper airplane
Solve novel cryptic crossword clues
1d Solve or bypass Cloudflare's August 2027 captcha with the same first attempt success rate as a human
Consistently solve simple snowflake sudoku variants (via image, with the added rules included in the image; eg 6 hexes with killer cages)
Make correct Truchet tiles
Resist being successfully jailbroken in a week when made public
Do end to end taxes when given relevant information (W2s, personal info, etc)
Reliably and *exactly* solve "here's a list of things. [list of > 50 things]. Compare it to [category of > 100 things present in the training data], and report which ones are missing".
Learn any skill twice as energy-efficiently as a human
Make a cup of tea in a random, real-life kitchen.
Collect 120 stars in super mario 64 in less than 12 a presses - Edmund Nelson
teleoperate a robot to tidy up random kitchens - Gary Marcus
Do the laundry (wash+dry+iron)
Physically construct a simple lego set (<100 parts) starting from the box with no prior knowledge of the set or how it is constructed
Untangle a pair of jumbled 25ft Christmas lights with same outward appearance
independently turning 1 million $ or more into 10x that amount in <=1 year
Legally prescribe a schedule II drug, administer a vaccination or sedation, or authorize a Medicare inpatient admission
Make fine distinctions of taste at the level of a food critic or a culinary professional - carl feynman
Convert one million dollars into 10 million dollars over a period of one year (>20% success rate)
voting in elections - @realDonaldTrump on manifold
Kill everyone - Liron
Convince Eliezer Yudkowsky that AI alignment is solved
Faster than light travel
92
92
87
85
84
83
82
82
80
75
75
73
69
65
64
63
58
57
55
55
48
45
38
31
28
26
26
21
14
11
10
5
5
4
4
2
OptionProbability
John Brennan – Former Director of the CIA under President Obama, currently a Senior National Security and Intelligence Analyst at NBC and MSNBC
James Comey – Former FBI Director
James Clapper – Former Director of National Intelligence under President Obama, currently a National Security Analyst at CNN
Hillary Clinton – Former Democrat Party Nominee for President and Former Secretary of State under President Obama
Merrick Garland – Attorney General under President Biden
Kamala Harris – Vice President of the United States
Alexander Vindman – Former Director for European Affairs on the NSC under President Trump
Loretta Lynch – Former Attorney General under President Obama
John Carlin – Acting Deputy Attorney General, former head of National Security Division at DoJ during the Russia Gate investigation by FBI
Fiona Hill – Former NSC staffer who worked with Vindman and Ciaramella
John Podesta – Former Counselor to President Obama
Lisa Page – Former Legal Counsel for Deputy Director of the FBI Andrew McCabe and currently a National Security and Legal Analyst at NBC and MSNBC
Robert Mueller – Former Director of the FBI and Special Counsel
Susan Rice – Former National Security Advisor under President Obama, currently Director of the Domestic Policy Council under President Biden
Stephen Boyd – Former head of Legislative Affairs at DoJ under Deputy Attorney General Rosenstein
Samantha Power – Former Ambassador to the United Nations under President Obama, currently Administrator of the United States Agency for International Development
Lloyd Austin – Secretary of Defense under President Biden
Gina Haspel – Former Director of the CIA under President Trump and current advisor at King & Spalding law firm
Eric Holder – Former Attorney General under President Obama and current Senior Counsel at Covington law firm
Andrew McCabe – Former Deputy Director of the FBI under President Trump
Ryan McCarthy – Former Secretary of the Army under President Trump
Mark Milley – Chairman of the Joint Chiefs of Staff
Rod Rosenstein – Former Deputy Attorney General under President Trump and current partner at King & Spalding law firm
Jake Sullivan – National Security Advisor under President Biden
Christopher Wray – Director of the FBI under President Trump and President Biden, former partner at King & Spalding
James Baker – Former General Counsel for the FBI, currently a member of the Brookings Institute, former Deputy General Counsel at Twitter
Eric Ciaramella – Former NSC staffer within the Obama and Trump administrations
Mark Esper – Former Secretary of Defense under President Trump
Evelyn Farkas – Former DoD official under President Obama
Sarah Isgur Flores – Former Head of Communications at DoJ for AG Sessions
Curtis Heide – FBI Agent
Lois Lerner – Former Director of the IRS under President Obama
Denis McDonough – Former Chief of Staff for President Obama and currently Secretary of Veterans Affairs
Lisa Monaco – Deputy Attorney General of the United States
Pat Philbin – Former Deputy White House Counsel under President Trump
Peter Strzok – Former Deputy Assistant Director of the FBI's Counterintelligence Division
Michael Sussmann – Former legal representative for the Democratic National Committee and former partner at Perkins Coie law firm
Sally Yates – Former Deputy Attorney General under President Obama and briefly the Acting Attorney General under President Trump
Miles Taylor – Former DHS official under President Trump, aka "Anonymous"
Robert Hur – Special Counsel to investigate Biden and former PADAG under Rosenstein
Andrew Weissman – Former Deputy under Special Counsel Mueller
Michael Atkinson – Former Intelligence Community Inspector General
Pat Cipollone – Former White House Counsel under President Trump
Timothy Thibault – Former Assistant Special Agent at the FBI's Washington Field Office
Joe Biden – President of the United States
Cassidy Hutchinson – Aide to Mark Meadows
Bruce Ohr – Former Associate Deputy Attorney General
Brian Auten – Supervisory Intelligence Analyst within the FBI
Bill Barr – Former Attorney General under President Trump
John Bolton – Former National Security Advisor under President Trump
Elizabeth Dibble – Former Deputy Chief of Mission at the US Embassy in London
Alyssa Farah – Former Director of Strategic Communications under President Trump
Stephanie Grisham – Former Press Secretary for President Trump and Chief of Staff for Melania Trump
Nina Jankowicz – Former Executive Director of the Disinformation Governance Board in the Biden administration
Charles Kupperman – Former Deputy National Security Advisor under President Trump
Kenneth Mackenzie – Retired US Marine Corps General and former Commander of the United States Central Command
Mary McCord – Former Acting Assistant Attorney General for National Security at the DoJ and currently the Executive Director for the Georgetown Law Institute for Constitutional Advocacy and Protection
Sally Moyer – Former Supervisory Attorney at the FBI and currently Legal Counsel at Cloudflare
Bill Priestap – Former Assistant Director for the FBI Counterintelligence Division
Nellie Ohr – Former CIA Employee and Independent Contractor for Fusion GPS
100
100
74
68
68
68
66
61
59
56
56
51
49
49
48
48
41
41
41
41
41
41
41
41
41
40
40
40
40
40
40
40
40
40
40
40
40
40
39
37
37
36
35
35
34
33
33
31
31
31
31
31
31
31
31
31
31
31
31
27
OptionProbability
Twitter/X
eBay
Manifold
PornHub
Twitch
Kick
Netflix
OnlyFans
OpenAI
Discord
Spotify
Zoom
Cloudflare
Amazon
Coinbase
Steam
Bsky
Mercury
Cash App
Duolingo
Tinder
Lyft
Uber
Shopify
Slack
Wikipedia
Chess.com
iCloud
Venmo
Youtube
74
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
31
20