Dario Amodei

AI models will probably score about 90% on the SWE-bench software-engineering benchmark by November 2025, possibly sooner.

“And I think in another year we'll probably be at 90%. I mean, I don't know, but might even be less than that.”

Said on

11 November 2024 · Lex Fridman Podcast #452; said right after noting models went from 3% to 50% on SWE-bench in 10 months

Source

Transcript for Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity · Lex Fridman Podcast

Should come true

Nov 2025

How sure they sounded

I think; probably; I don't know

Categories

AI

Importance

2 of 5 · A benchmark milestone that matters mainly to the AI field.

Outcome

Right, but late

The best SWE-bench Verified score by the end of November 2025 was Claude Opus 4.5's, released Nov 24, 2025 and described as state of the art at roughly 81%, well short of 90%. Reports put the top score near 87.6% in April 2026, and the BenchLM leaderboard (July 2026) shows several models above 90%. So the milestone came after the one-year window. These are mostly aggregator figures, and the benchmark has contamination caveats.

Evidence:

• https://www.anthropic.com/news/claude-opus-4-5
• https://benchlm.ai/benchmarks/swe-verified
• https://codesota.com/agentic/best-for-swe-bench