Training Data News

What’s the least impressive thing you’re very sure AI still won’t be able to do before August 2027? [read description]

Jul 20, 4:26 AMJul 20, 4:15 AM

28743425

OptionProbability

Independently turning 1 thousand $ or more into 1.2x that amount in one year

Have human conversations that feel natural (the human knows it's an AI)

Beat a mainline Pokémon game, glitchless, with no more assistance than ClaudePlaysPokemon, in a month of compute time

Recognize sarcasm as well as a typical human

Book airline tickets from simple instructions (from/to, dates/time, class, price, payment information)

Write an essay on a highschool-level topic that doesn't have "AI-generated" vibes

Generating labeled diagrams of some arbitrary device(s) (within reason)

Reliably follow an instruction for the duration of a long conversation without the instruction being reiterated

Predict future better than human experts in some area of forecasting (eg politics, sports, technology)

Solve novel cryptic crossword clues

Name every [metro system] station whose name contains/doesn't contain [letter or letters], with >95% accuracy (excluding weird edge cases like stations with multiple names)

Solve intermediate no-guess minesweeper boards at least 80% of the time

Do end to end taxes when given relevant information (W2s, personal info, etc)

Consistently solve simple snowflake sudoku variants (via image, with the added rules included in the image; eg 6 hexes with killer cages)

Write a somewhat original, full length, screen-play with a coherent story, with no plot or continuity errors.

Consistently stop hallucinating after being corrected by the user

Make correct Truchet tiles

1d Solve or bypass Cloudflare's August 2027 captcha with the same first attempt success rate as a human

Consistently and correctly answer prompts of the format: "How many times does the word [word] occur in the following text: [~10000 words]" without writing and executing code or utilising any other external tools

Stop making any obvious mistakes (e.g. strawberry, 9.11>9.9)

Fold a paper airplane

Learn any skill twice as energy-efficiently as a human

Reliably and *exactly* solve "here's a list of things. [list of > 50 things]. Compare it to [category of > 100 things present in the training data], and report which ones are missing".

Make a cup of tea in a random, real-life kitchen.

Resist being successfully jailbroken in a week when made public

Do the laundry (wash+dry+iron)

teleoperate a robot to tidy up random kitchens - Gary Marcus

Collect 120 stars in super mario 64 in less than 12 a presses - Edmund Nelson

Untangle a pair of jumbled 25ft Christmas lights with same outward appearance

Kettle-stitching an antiquarian book - Hilarius Bookbinder

Physically construct a simple lego set (<100 parts) starting from the box with no prior knowledge of the set or how it is constructed

Legally prescribe a schedule II drug, administer a vaccination or sedation, or authorize a Medicare inpatient admission

independently turning 1 million $ or more into 10x that amount in <=1 year

Make fine distinctions of taste at the level of a food critic or a culinary professional - carl feynman

Convert one million dollars into 10 million dollars over a period of one year (>20% success rate)

voting in elections - @realDonaldTrump on manifold

Convince Eliezer Yudkowsky that AI alignment is solved

Faster than light travel

Kill everyone - Liron

Which of these outcomes of the NYT vs. OpenAI lawsuit will occur?

Dec 29, 1:50 AMJan 1, 4:59 AM

21329858

OptionProbability

The case settles for more than $10 million before a verdict is reached.

The case settles before a verdict is reached.

New York Times wins at least $1 in damages via a verdict.

Disney sues OpenAI for copyright infringement before the NYT case concludes.

NYT and OpenAI announce a partnership that goes beyond access to training data, such as the one OpenAI made with Politico.

The case appears before the Supreme Court.

Nintendo sues OpenAI for copyright infringement before the NYT case concludes.

The case is fully resolved somehow, including all appeals, by the end of 2026.

OpenAI wins via an outright verdict in their favor.

The case settles for more than $100 million before a verdict is reached.

New York Times wins at least $10 million in damages via a verdict.

New York Times wins at least $100 million in damages via a verdict.

New York Times wins at least $1 billion in damages via a verdict.

The Supreme Court rules in favor of The New York Times and upholds damages and compensation with a net present value of $10 million or more, but does not order GPT-4 deleted.

The Supreme Court orders GPT-4 to be deleted, or OpenAI otherwise agrees to delete GPT-4 on the basis of this case.

New York Times wins a verdict ordering GPT-4 to be deleted.

New York Times wins a verdict ordering GPT-4 to be deleted from the Supreme Court.

The case is fully resolved somehow, including all appeals, by the end of 2025.

The case is fully resolved somehow, including all appeals, by the end of 2024.

LLM trained on data from 1900 comes up with special relativity from scratch by the end of what year

Oct 5, 9:45 PMJan 1, 11:59 PM

182679

OptionProbability

2035

2030

2025

What will happen with the recent lawsuits by major record labels against AI music startups?

Jun 26, 12:43 AMJan 1, 7:59 AM

111990

OptionProbability

A lawsuit reaches a settlement

Training data is leaked or disclosed publicly by the court

Microsoft drops its partnership with Suno

A lawsuit reaches judgment

A lawsuit reaches trial

A lawsuit is dismissed

A lawsuit reaches judgment in favor of the plaintiff

A lawsuit expands to include Microsoft as a defendant

A lawsuit reaches judgment in favor of the defendant

Udio or Suno will shutdown services permanently

A Billboard Top-100 Artist testifies at trial

Udio or Suno will shutdown services temporarily

100

What will be true about the first AGI that becomes accessible to everyone? [ADD MORE]

Apr 24, 12:58 PMMar 22, 10:59 PM

351938

OptionProbability

Runs entirely on silicon-based non-quantum machines

The existence of the AGI and the model name were leaked prior to release and official announcement

Developed by a US company

Trained with synthetic data and self-play

Doomsday clock moved at least one full minute closer to midnight due to this AI

Transformer based or mix of Transformer and other architectures

Has an internal World Model able to simulate basic physics without external tools

Not available in EU countries initially for the first 4 months after public release

Based on/inspired by algorithms from the human brain (beyond neural networks).

Public announcement was made after 1 year or more of finishing training

Solves at least one of the seven Millennium Problems

Not a static model; weights change during inference

Used by a Figure humanoid robot

Infinite context window

Based on/inspired by OpenAI's Sora (resolves YES if explicitly stated by developers in announcement or paper)

This market will resolve N/A

Is OpenAI's GPT-6, whatever name or architecture it has

No consensus that it's AGI until over a month after being announced and interacting with people outside the lab.

Developed by a Chinese company

Open sourced

Developed by the open-source community instead of a company

Developed by an EU company

Is OpenAI's GPT-5, whatever name or architecture it has

How Will the LLM Hallucination Problem Be Solved?

Mar 31, 8:31 PMJan 1, 5:59 AM

32977

OptionProbability

It won't be

Giving all LLMs access to the internet and databases of scientific papers

Other

Bigger model trained on more data + RL

Vector Embeddings (as with Pinecone https://www.pinecone.io/)

Filtering (as with Deepmind AlphaCode https://www.deepmind.com/blog/competitive-programming-with-alphacode)

Feed Forward Algorithms (https://www.cs.toronto.edu/~hinton/FFA13.pdf)

Vigger models + prompt engineering

Ensemble Combined with Fine Tuning

Joint Embedding Predictive Architecture (https://arxiv.org/pdf/2301.08243.pdf)

Will an AI be successfully trained to play Populous The Beginning multiplayer based on bottom-up data (like video training data) by the year 2030?

Jan 1, 7:33 AMNov 1, 5:01 AM

69.04%chance

40901

OptionVotes

YES

1091

946

Will any widely used LLM be pre-trained with abstract synthetic data before 2030?

Sep 1, 6:48 AMJan 2, 2:59 AM

72.49%chance

4775

OptionVotes

YES

1623

616

We are going to start running out of data to train large language models in [YEAR]

Oct 14, 3:57 PMFeb 2, 6:29 PM

9336

OptionProbability

2025

2026

2030

2024

2028

2029

2027

2023

Leading AI lab spends $100k+ on remote worker training data before 2028?

Apr 20, 3:56 PMJan 1, 11:59 PM

86.02%chance

13293

OptionVotes

YES

248

Will manifold be part of GPT5's training data?

Sep 21, 8:21 PMJan 1, 4:59 AM

76%chance

578

OptionVotes

YES

188

101

Will Japan prevent AI training on copyrighted data before 2030?

Jun 1, 2:32 AMJan 1, 8:00 AM

33.39%chance

475

OptionVotes

YES

116

Training Data News

Prediction markets for Training Data

Prediction markets for Training Data

Prediction markets for Training Data

What’s the least impressive thing you’re very sure AI still won’t be able to do before August 2027? [read description]

Which of these outcomes of the NYT vs. OpenAI lawsuit will occur?

LLM trained on data from 1900 comes up with special relativity from scratch by the end of what year

What will happen with the recent lawsuits by major record labels against AI music startups?

What will be true about the first AGI that becomes accessible to everyone? [ADD MORE]

How Will the LLM Hallucination Problem Be Solved?

Will an AI be successfully trained to play Populous The Beginning multiplayer based on bottom-up data (like video training data) by the year 2030?

Will any widely used LLM be pre-trained with abstract synthetic data before 2030?

We are going to start running out of data to train large language models in [YEAR]

Leading AI lab spends $100k+ on remote worker training data before 2028?

Will manifold be part of GPT5's training data?

Will Japan prevent AI training on copyrighted data before 2030?

Latest stories