SWE Bench News

85% - 90%

75% - 85%

90% - 100%

70% - 75%

53% - 60%

X = 2

X = 4

X = 8

X = 16

X = 32

X = 60

X = 40

X = 50

X = 75

X = 85

X = 70

X = 90

X = 80

X = 5

X = 10

X = 15

X = 20

X = 30

X = 50

X = 65

X = 40

X = 60

X = 70

X = 75

X = 85

>85

70-85 inclusive

NO

YES

2025

2026

2027

2028

2029

later

NO

It will allow AI models to hallucinate much less

It will allow AI models to become much better at working with long-context

It will allow AI models to use long context more efficiently

It will make AI models better at making art (creative writing, visual art)

It will make make AI models better at visual reasoning

it will make AI models better at agentic tasks (such as OSWorld and SWE-bench)

It will make AI models much better at answering competitive math questions