benchmark evidence
SWE-bench Verified
Real-world GitHub issue resolution benchmark.
winner on SWE-bench Verified
direct benchmark result, not a broad vertical composite | source row dated 2000-01-01
scored on 2026-04-15 · stale source data (45d)
latest mapped results | top 20
| # | Model | Score | Evidence | Tested |
|---|---|---|---|---|
| 1 | Anthropic: Claude Opus 4.6 | 80.8 | model-only independent_benchmark | 2000-01-01 |
| 2 | DeepSeek: DeepSeek V4 Pro | 80.6 | model-only independent_benchmark | 2000-01-01 |
| 3 | OpenAI: GPT-5.2 | 80.0 | model-only independent_benchmark | 2000-01-01 |
| 4 | DeepSeek: DeepSeek V4 Flash | 79.0 | model-only independent_benchmark | 2000-01-01 |
| 5 | Z.ai: GLM 5 | 77.8 | model-only independent_benchmark | 2000-01-01 |
| 6 | OpenAI: GPT-5.1 | 76.3 | model-only independent_benchmark | 2000-01-01 |
| 7 | Anthropic: Claude Opus 4.5 | 76.0 | agent-dependent independent_benchmark | mini-swe-agent | 2026-04-15 |
| 8 | Anthropic: Claude Haiku 4.5 | 73.3 | model-only independent_benchmark | 2000-01-01 |
| 9 | DeepSeek: DeepSeek V3.2 | 73.1 | model-only independent_benchmark | 2000-01-01 |
| 10 | Anthropic: Claude Sonnet 4.5 | 72.0 | agent-dependent independent_benchmark | mini-swe-agent | 2026-04-15 |
| 11 | MoonshotAI: Kimi K2 Thinking | 71.3 | model-only independent_benchmark | 2000-01-01 |
| 12 | Anthropic: Claude Opus 4.7 | 69.0 | agent-dependent independent_benchmark | SWE-agent | 2026-03-15 |
| 13 | OpenAI: GPT-5 | 68.0 | agent-dependent independent_benchmark | mini-swe-agent | 2026-03-15 |
| 14 | Anthropic: Claude Sonnet 4.6 | 64.0 | agent-dependent independent_benchmark | mini-swe-agent | 2026-03-15 |
| 15 | OpenAI: o3 | 61.0 | agent-dependent independent_benchmark | mini-swe-agent | 2026-02-15 |
| 16 | OpenAI: GPT-4.1 | 59.6 | agent-dependent independent_benchmark | SWE-agent | 2026-04-15 |
| 17 | OpenAI: o4 Mini | 57.0 | agent-dependent independent_benchmark | mini-swe-agent | 2026-02-15 |
| 18 | OpenAI: o1 | 52.0 | agent-dependent independent_benchmark | SWE-agent | 2025-09-15 |
| 19 | DeepSeek: R1 | 49.0 | agent-dependent independent_benchmark | OpenHands | 2026-01-15 |
| 20 | Google: Gemini 2.5 Pro | 48.0 | agent-dependent independent_benchmark | SWE-agent | 2025-09-15 |
what this result means
Real-world GitHub issue resolution benchmark.
This benchmark contributes direct public evidence. Read its scope before generalizing the result.
A win here is a win on SWE-bench Verified. Broad task pages require independent corroboration before naming a general winner.
source record
category: coding
metric: resolved_percent
matched models: 21
latest source date: 2026-04-15
direction: higher is better