Model Tracker
OTIS Mock AIME 2024–2025 leaderboard
The models in the Model Tracker that Epoch AI has measured on OTIS Mock AIME 2024–2025, ranked by the highest score of each: the name Epoch AI gives it, the setting it ran and the day.
En inglés
The models in the Model Tracker that Epoch AI has measured on OTIS Mock AIME 2024–2025, ranked by the highest score of each: the name Epoch AI gives it, the setting it ran and the day. Epoch AI, ‘Capabilities & benchmarking’, epoch.ai/benchmarks, CC BY 4.0: Epoch AI’s own runs and its Capabilities Index.
- 1. Gemini 4 Argon: 100.0% (± 0.0) — gemini-4-argon_high, high, 13 October 2026
- 1. GPT-6.1 Sol: 100.0% (± 0.0) — gpt-6.1-sol_max, max, 29 September 2026
- 1. Claude Sonnet 5.5: 100.0% (± 0.0) — claude-sonnet-5-5_max, max, 29 September 2026
- 1. GPT-6 Sol: 100.0% (± 0.0) — gpt-6-sol_max, max, 22 September 2026
- 1. Claude Opus 5.5: 100.0% (± 0.0) — claude-opus-5-5_max, max, 22 September 2026
- 1. Claude Fable 5.1: 100.0% (± 0.0) — claude-fable-5-1_max, max, 1 September 2026
- 1. GPT-6 Astra: 100.0% (± 0.0) — gpt-6-astra_max, max, 30 August 2026
- 1. GPT-5.6 Sol: 100.0% (± 0.0) — gpt-5.6-sol_max, max, 9 July 2026
- 1. Claude Fable 5: 100.0% (± 0.0) — claude-fable-5_high, high, 6 August 2026
- 1. GPT-5.5: 100.0% (± 0.0) — gpt-5.5-pre-release_xhigh, xhigh, 24 April 2026
- 1. GPT-5.5 Pro: 100.0% (± 0.0) — gpt-5.5-pro-pre-release_xhigh, xhigh, 24 April 2026
- 12. GPT-5.6 Terra: 99.7% (± 0.3) — gpt-5.6-terra_max, max, 9 July 2026
- 12. Claude Fable 5.2: 99.7% (± 0.3) — claude-fable-5-2_max, max, 6 November 2026
- 14. Muse Wonder: 99.4% (± 0.4) — muse-wonder-1.0_max, max, 12 October 2026
- 14. Qwen3.8-Max: 99.4% (± 0.4) — qwen3.8-max_xhigh, xhigh, 4 August 2026
- 14. Mistral Large 4: 99.4% (± 0.4) — mistral-large-4-0-oss_high, high, 2 November 2026
- 17. Grok 4.6: 99.2% (± 0.5) — grok-4.6_xhigh, xhigh, 14 August 2026
- 18. Claude Haiku 5.5: 98.9% (± 0.5) — claude-haiku-5-5_xhigh, xhigh, 7 October 2026
- 18. GPT-6 Luna: 98.9% (± 0.7) — gpt-6-luna_max, max, 22 September 2026
- 18. Gemini 3.8 Flash: 98.9% (± 0.9) — gemini-3.8-flash_high, high, 2 September 2026
- 18. Claude Opus 5: 98.9% (± 1.1) — claude-opus-5_max, max, 24 July 2026
- 22. DeepSeek-V4.1-Flash: 98.3% (± 0.6) — deepseek-v4.1-flash_max, max, 8 October 2026
- 22. GPT-5.6 Luna: 98.3% (± 1.2) — gpt-5.6-luna_max, max, 9 July 2026
- 22. Claude Opus 4.8: 98.3% (± 1.4) — claude-opus-4-8_max, max, 7 June 2026
- 25. Grok 4.7: 98.1% (± 1.4) — grok-4.7_xhigh, xhigh, 22 September 2026
- 26. Claude Opus 4.7: 97.8% (± 2.2) — claude-opus-4-7_xhigh, xhigh, 17 April 2026
- 27. Grok 4.5: 97.8% (± 1.3) — grok-4.5_high, high, 8 July 2026
- 27. GPT-5.4: 97.8% (± 2.2) — gpt-5.4-2026-03-05_high, high, 15 July 2026
- 29. Gemini 3.7 Flash: 97.2% (± 1.9) — gemini-3.7-flash_high, high, 14 August 2026
- 30. DeepSeek V4 Pro: 96.7% (± 2.0) — deepseek-v4-pro_max, max, 17 June 2026
- 31. GPT-5.2: 96.1% (± 2.6) — gpt-5.2-2025-12-11_high, high, 11 December 2025
- 32. Gemini 3.1 Pro: 95.6% (± 3.1) — gemini-3.1-pro-preview, 20 February 2026
- 33. Gemini 3.5 Flash: 95.6% (± 2.7) — gemini-3.5-flash_high, high, 25 May 2026
- 33. Gemini 3 Flash: 95.6% (± 3.1) — gemini-3-flash-preview_high, high, 6 August 2026
- 35. Claude Sonnet 5: 94.7% (± 2.3) — claude-sonnet-5_xhigh, xhigh, 1 July 2026
- 36. Claude Opus 4.6: 94.4% (± 2.8) — claude-opus-4-6_64K, 64K, 6 February 2026
- 37. Gemini 3.6 Flash: 94.2% (± 3.1) — gemini-3.6-flash_high, high, 2 August 2026
- 38. Grok 4.20: 92.2% (± 3.3) — grok-4.20-0309-reasoning, 13 July 2026
- 39. GPT-5: 91.4% (± 3.7) — gpt-5-2025-08-07_high, high, 29 October 2025
- 39. Gemini 3 Pro: 91.4% (± 3.7) — gemini-3-pro-preview, 19 November 2025
- 41. Qwen3.6-27B: 91.1% (± 4.3) — qwen3.6-27b, 7 August 2026
- 42. Muse Spark: 88.9% (± 4.7) — muse-spark, 8 April 2026
- 43. gpt-oss-120b: 88.9% (± 4.5) — openai/gpt-oss-120b_high, high, 11 December 2025
- 43. GPT-5.4 Mini: 88.9% (± 4.7) — gpt-5.4-mini-2026-03-17_xhigh, xhigh, 7 August 2026
- 45. GPT-5.1: 88.6% (± 4.0) — gpt-5.1-2025-11-13_high, high, 13 November 2025
- 46. GPT-5.4 nano: 87.8% (± 3.9) — gpt-5.4-nano-2026-03-17_high, high, 14 April 2026
- 47. Qwen3.6-35B-A3B: 86.7% (± 5.1) — qwen3.6-35b-a3b, 7 August 2026
- 47. GPT-5 Mini: 86.7% (± 4.3) — gpt-5-mini-2025-08-07_high, high, 30 October 2025
- 49. Claude Opus 4.5: 86.1% (± 4.3) — claude-opus-4-5-20251101_32K, 32K, 24 November 2025
- 50. Claude Sonnet 4.6: 85.8% (± 4.5) — claude-sonnet-4-6_32K, 32K, 20 February 2026
- 51. Gemini 2.5 Pro: 84.7% (± 4.8) — gemini-2.5-pro, 16 November 2025
- 52. o3: 84.4% (± 5.5) — o3-2025-04-16_medium, medium, 7 August 2026
- 53. o4-mini: 81.7% (± 4.7) — o4-mini-2025-04-16_high, high, 16 April 2025
- 54. GPT-5 nano: 81.1% (± 4.6) — gpt-5-nano-2025-08-07_high, high, 31 October 2025
- 55. Gemini 3.1 Flash-Lite: 80.0% (± 6.0) — gemini-3.1-flash-lite_high, high, 6 August 2026
- 56. Claude Sonnet 4.5: 77.8% (± 6.3) — claude-sonnet-4-5-20250929_59K, 59K, 28 October 2025
- 56. Grok 3 mini: 77.8% (± 6.3) — grok-3-mini-beta_high, high, 10 April 2025
- 58. o3-mini: 76.9% (± 5.2) — o3-mini-2025-01-31_high, high, 27 February 2025
- 59. o1: 73.3% (± 6.7) — o1-2024-12-17_high, high, 15 July 2026
- 60. Gemini 2.5 Flash: 73.1% (± 5.5) — gemini-2.5-flash-preview-04-17, 22 April 2025
- 61. Gemini 3.5 Flash-Lite: 71.1% (± 6.8) — gemini-3.5-flash-lite_high, high, 6 August 2026
- 61. Claude Sonnet 4: 71.1% (± 6.8) — claude-sonnet-4-20250514_32K, 32K, 22 May 2025
- 63. Qwen3.5-35B-A3B: 70.0% (± 6.0) — qwen3.5-35b-a3b_none, none, 30 August 2026
- 64. Claude Opus 4.1: 68.9% (± 7.0) — claude-opus-4-1-20250805_27K, 27K, 5 August 2025
- 65. Qwen3-32B: 66.9% (± 6.0) — Qwen3-32B, 30 August 2026
- 66. Claude Haiku 4.5: 66.7% (± 7.1) — claude-haiku-4-5-20251001_32K, 32K, 22 October 2025
- 67. Qwen3-14B: 66.4% (± 6.2) — qwen3-14b, 28 August 2026
- 67. DeepSeek-R1: 66.4% (± 4.9) — DeepSeek-R1-0528, 29 May 2025
- 69. gpt-oss-20b: 65.3% (± 5.9) — gpt-oss-20b_medium, medium, 27 August 2026
- 70. Qwen3-30B-A3B: 62.8% (± 6.2) — Qwen3-30B-A3B, 28 August 2026
- 71. Qwen3.5-9B: 61.7% (± 6.0) — qwen3.5-9B_none, none, 27 August 2026
- 72. Claude Sonnet 3.7: 57.8% (± 7.4) — claude-3-7-sonnet-20250219_64K, 64K, 13 March 2025
- 73. Qwen3-8B: 56.1% (± 6.5) — qwen3-8b, 27 August 2026
- 74. Qwen3.5-4B: 55.8% (± 6.2) — qwen3.5-4b_none, none, 28 August 2026
- 75. Grok 3: 55.6% (± 7.5) — grok-3-beta, 10 April 2025
- 76. o1-mini: 46.9% (± 6.5) — o1-mini-2024-09-12_high, high, 6 March 2025
- 77. GPT-4.1 mini: 44.7% (± 6.1) — gpt-4.1-mini-2025-04-14, 14 April 2025
- 78. GPT-4.1: 38.3% (± 6.3) — gpt-4.1-2025-04-14, 14 April 2025
- 79. GPT-4.5: 37.8% (± 7.3) — gpt-4.5-preview-2025-02-27, 28 February 2025
- 79. DeepSeek-V3: 37.8% (± 6.2) — DeepSeek-V3-0324, 1 April 2025
- 81. Mistral Medium 3: 32.2% (± 6.0) — mistral-medium-2505, 7 May 2025
- 82. o1-preview: 31.1% (± 7.0) — o1-preview-2024-09-12, 7 March 2025
- 82. Gemini 2.0 Flash: 31.1% (± 6.3) — gemini-2.0-flash-001, 25 February 2025
- 84. Mistral Small 3.2: 30.3% (± 5.8) — mistral-small-3.2-2506, 30 August 2026
- 85. Magistral Small 1.0: 30.0% (± 5.9) — magistral-small-2506, 6 August 2026
- 86. GPT-4.1 nano: 28.9% (± 6.0) — gpt-4.1-nano-2025-04-14, 14 April 2025
- 87. Magistral Small 1.2: 28.1% (± 5.6) — magistral-small-2509, 30 August 2026
- 88. Gemini 1.5 Pro: 23.1% (± 5.2) — gemini-1.5-pro-002, 25 February 2025
- 89. Llama 4 Maverick: 20.6% (± 4.9) — Llama-4-Maverick-17B-128E-Instruct-FP8, 8 April 2025
- 90. Gemini 1.5 Flash: 16.3% (± 4.1) — gemini-1.5-flash-002, 25 February 2025
- 91. Grok-2: 11.5% (± 3.4) — grok-2-1212, 25 February 2025
- 92. Qwen3-1.7B: 8.1% (± 2.8) — qwen3-1.7b_none, none, 28 August 2026
- 93. Llama 4 Scout: 7.8% (± 3.0) — Llama-4-Scout-17B-16E-Instruct, 8 April 2025
- 94. GPT-4o mini: 6.9% (± 2.7) — gpt-4o-mini-2024-07-18, 30 July 2025
- 95. GPT-4 Turbo: 6.7% (± 2.4) — gpt-4-turbo-2024-04-09, 27 February 2025
- 96. GPT-4o: 6.4% (± 2.6) — gpt-4o-2024-08-06, 25 February 2025
- 97. Mistral Small 3.1: 5.8% (± 2.3) — mistral-small-2503, 18 March 2025
- 98. Claude 3 Opus: 4.7% (± 2.0) — claude-3-opus-20240229, 25 February 2025
- 99. Gemini 1.5 Flash-8B: 4.6% (± 2.0) — gemini-1.5-flash-8b-001, 25 February 2025
- 100. Claude 3.5 Haiku: 4.3% (± 1.6) — claude-3-5-haiku-20241022, 25 February 2025
- 101. Claude 2: 2.5% (± 1.5) — claude-2.0, 7 March 2025
- 101. Claude 3 Sonnet: 2.5% (± 2.0) — claude-3-sonnet-20240229, 25 February 2025
- 103. GPT-3.5 Turbo: 2.2% (± 2.2) — gpt-3.5-turbo-0125, 7 August 2026
- 104. Claude 2.1: 1.9% (± 1.0) — claude-2.1, 7 March 2025
- 105. Claude 3 Haiku: 1.8% (± 0.8) — claude-3-haiku-20240307, 25 February 2025
- 106. GPT-4: 1.1% (± 0.5) — gpt-4-0613, 23 October 2025
Epoch AI: the benchmark and its data · CC BY 4.0