SWE Bench News

X = 5

X = 10

X = 15

X = 20

X = 30

X = 50

X = 60

X = 70

X = 40

X = 65

X = 75

X = 80

X = 2

X = 4

X = 8

X = 16

X = 32

X = 60

X = 40

X = 50

X = 75

X = 85

X = 70

X = 90

X = 80

uses some non-AdamW optimizer

Some image input (multimodality)

>= 50% on TerminalBench (https://www.tbench.ai/leaderboard)

>= 60% on BrowseComp (https://www.kaggle.com/benchmarks/openai/browsecomp)

1M+ Context

>=25T pretraining tokens

>= 73% on SWE-Bench Verified (according to epoch.ai)

>= 2 shared experts

Some variation of NSA (Native Sparse Attention)

>=16 active experts

Gemini 2.5 Pro tier or higher on FictionBench (90.6%+ at 192k)

>=512 experts

intra-expert communication

DS-MoE with adaptative expert count

>= 44% on Humanity's Last Exam (text only) at scale.com leaderboard

DeepSeek reports some results with a full-blown deep research agent, and emphasizes that this is the intended use-mode

>=52B active parameters

>=1.5T parameters

NO

70 - 79%

80 - 89%

60 - 69%

90 - 100%

50 - 59%

60-70%

Other

50-60%

YES

2026

2027

2028

2029

2030

2031

2032

2026

2027

2028

2029

later

2024

YES

71 - 82%

83 - 94%

59 - 70%

23 - 34%

35 - 46%

47 - 58%

95 - 100%

it will make AI models better at agentic tasks (such as OSWorld and SWE-bench)

It will allow AI models to become much better at working with long-context

It will allow AI models to learn at test time

It will allow AI models to hallucinate much less

It will allow AI models to use long context more efficiently

It will make make AI models better at visual reasoning

It will make AI models much better at answering competitive math questions