Vai al contenuto
AI.info

Model Tracker

SWE-bench Verified leaderboard

The models in the Model Tracker that Epoch AI has measured on SWE-bench Verified, ranked by the highest score of each: the name Epoch AI gives it, the setting it ran and the day. Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks,

In inglese

The models in the Model Tracker that Epoch AI has measured on SWE-bench Verified, ranked by the highest score of each: the name Epoch AI gives it, the setting it ran and the day. Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks, CC BY 4.0: Epoch AI’s own runs and its Capabilities Index.

  1. 1. Claude Opus 4.7: 83.5% (± 1.7) — claude-opus-4-7_max, max, 20 April 2026
  2. 2. GPT-5.5: 80.6% (± 1.8) — gpt-5.5-pre-release_xhigh, xhigh, 24 April 2026
  3. 3. Gemini 3.5 Flash: 79.3% (± 1.8) — gemini-3.5-flash_high, high, 1 June 2026
  4. 4. Claude Opus 4.6: 78.7% (± 1.9) — claude-opus-4-6, 18 February 2026
  5. 5. DeepSeek V4 Pro: 77.6% (± 1.9) — deepseek-v4-pro_max, max, 18 June 2026
  6. 6. GPT-5.4: 76.9% (± 1.9) — gpt-5.4-2026-03-05_high, high, 6 March 2026
  7. 7. Claude Opus 4.5: 76.7% (± 1.9) — claude-opus-4-5-20251101, 5 February 2026
  8. 8. Gemini 3.1 Pro: 75.6% (± 2.0) — gemini-3.1-pro-preview-customtools, 24 February 2026
  9. 9. Gemini 3 Flash: 75.4% (± 2.0) — gemini-3-flash-preview, 18 February 2026
  10. 10. Claude Sonnet 4.6: 75.2% (± 2.0) — claude-sonnet-4-6, 21 February 2026
  11. 11. GPT-5.3-Codex: 74.8% (± 2.0) — gpt-5.3-codex_high, high, 25 February 2026
  12. 12. GPT-5.2: 73.8% (± 2.0) — gpt-5.2-2025-12-11_high, high, 12 February 2026
  13. 13. GPT-5: 73.6% (± 2.0) — gpt-5-2025-08-07_high, high, 6 February 2026
  14. 14. Claude Opus 4.1: 73.3% (± 2.0) — claude-opus-4-1-20250805, 11 February 2026
  15. 15. Gemini 3 Pro: 72.9% (± 2.0) — gemini-3-pro-preview, 13 February 2026
  16. 16. Claude Sonnet 4.5: 71.3% (± 2.1) — claude-sonnet-4-5-20250929, 5 February 2026
  17. 17. GPT-5.1: 68.0% (± 2.1) — gpt-5.1-2025-11-13_high, high, 18 February 2026
  18. 18. GPT-5 Mini: 64.7% (± 2.2) — gpt-5-mini-2025-08-07_medium, medium, 1 February 2026
  19. 19. o3: 62.3% (± 2.2) — o3-2025-04-16_medium, medium, 12 February 2026
  20. 20. Claude Sonnet 3.7: 61.0% (± 2.2) — claude-3-7-sonnet-20250219, 4 February 2026
  21. 21. Gemini 2.5 Pro: 57.6% (± 2.3) — gemini-2.5-pro, 13 February 2026
  22. 22. GPT-4.1: 48.5% (± 2.3) — gpt-4.1-2025-04-14, 8 February 2026
  23. 23. GPT-4o: 31.0% (± 2.1) — gpt-4o-2024-11-20, 11 February 2026

Epoch AI: the benchmark and its data · CC BY 4.0

Other benchmarks

All of the Model Tracker