| #2 | gpt-4o-2024-05-13 Strong on RepoQA Official Results overall_average_pass_at_1_pct (99%) and RepoQA Official Results all_average_pass_at_1_pct (99%) | | 26.3% | 9 | RepoQA Official Results overall_average_pass_at_1_pct (Mar 12, 2026) RepoQA Official Results all_average_pass_at_1_pct (Mar 12, 2026) |
| #3 | z-ai/glm-4.7 Strong on Sonar Java Quality Leaderboard functional_skill_pct (74%) and Vals LiveCodeBench overall_accuracy_pct (91%) | | 29.7% | 16 | Sonar Java Quality Leaderboard functional_skill_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #5 | Kimi K2 Thinking | | 29.1% | 15 | Sonar Java Quality Leaderboard functional_skill_pct (Mar 12, 2026) Sonar Java Quality Leaderboard issue_density_error_per_kloc (Mar 12, 2026) |
| #6 | minimax/minimax-m2.1 | | 29.7% | 16 | Sonar Java Quality Leaderboard functional_skill_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #10 | deepseek/deepseek-r1 | | 25.3% | 17 | Sonar Java Quality Leaderboard functional_skill_pct (Mar 12, 2026) Aider Polyglot Leaderboard percent_correct_pct (Mar 12, 2026) |
| #11 | gemini-3-pro-preview | | 20.6% | 21 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #13 | google/gemini-3.1-pro-preview | | 18.1% | 16 | Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) Vals Terminal-Bench 2 overall_accuracy_pct (Mar 12, 2026) |
| #14 | Grok-4-0709 | | 24.4% | 18 | Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) Galileo Agent Leaderboard v2 Avg AC (Mar 12, 2026) |
| #15 | gpt-4.1-20250414 | | 24.2% | 18 | Galileo Agent Leaderboard v2 Avg AC (Mar 12, 2026) Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) |
| #17 | openai/gpt-5.4-2026-03-05 | | 17.5% | 15 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #19 | claude-sonnet-4-20250514 | | 21.3% | 17 | Galileo Agent Leaderboard v2 Avg AC (Mar 12, 2026) Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) |
| #20 | anthropic/claude-sonnet-4.6 | | 17.5% | 15 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #21 | anthropic/claude-opus-4-6-thinking | | 16.2% | 13 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #22 | Meta-Llama-3-70B-Instruct | | 17.1% | 4 | RepoQA Official Results overall_average_pass_at_1_pct (Mar 12, 2026) RepoQA Official Results all_average_pass_at_1_pct (Mar 12, 2026) |
| #23 | claude-opus-4-5-20251101 | | 17.9% | 16 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals Terminal-Bench 2 overall_accuracy_pct (Mar 12, 2026) |
| #24 | GLM-5 | | 16.0% | 7 | Sonar Java Quality Leaderboard functional_skill_pct (Mar 12, 2026) Sonar Java Quality Leaderboard issue_density_error_per_kloc (Mar 12, 2026) |
| #25 | gemini-3-flash-preview | | 17.3% | 15 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #27 | gpt-5-2025-08-07 | | 17.9% | 16 | Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) |
| #29 | gpt-5.2-2025-12-11 | | 16.2% | 13 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #30 | gpt-5.1-2025-11-13 | | 17.9% | 16 | Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) |
| #31 | anthropic/claude-opus-4-5-20251101-thinking | | 16.2% | 13 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #33 | gpt-4o | | 25.5% | 14 | Sonar Java Quality Leaderboard functional_skill_pct (Mar 12, 2026) MEGA-Bench app:Coding (Mar 12, 2026) |
| #34 | kimi/kimi-k2.5-thinking | | 17.3% | 15 | Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) |
| #37 | Meta-Llama-3-8B-Instruct | | 20.4% | 6 | RepoQA Official Results overall_average_pass_at_1_pct (Mar 12, 2026) RepoQA Official Results all_average_pass_at_1_pct (Mar 12, 2026) |
| #39 | anthropic/claude-sonnet-4-5-20250929-thinking | | 16.2% | 13 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #40 | gpt-4o-2024-08-06 | | 24.2% | 15 | Aider Code Editing Leaderboard percent_correct_pct (Mar 12, 2026) Aider Code Editing Leaderboard correct_edit_format_pct (Mar 12, 2026) |
| #41 | gemini-2.5-pro | | 19.6% | 21 | Galileo Agent Leaderboard v2 Avg AC (Mar 12, 2026) Vals Terminal-Bench 2 overall_accuracy_pct (Mar 12, 2026) |
| #43 | zai/glm-5-thinking | | 14.9% | 11 | Vals SWE-bench overall_accuracy_pct (Mar 12, 2026) Vals LiveCodeBench overall_accuracy_pct (Mar 12, 2026) |
| #44 | qwen-2.5-72b-instruct | | 16.4% | 10 | Galileo Agent Leaderboard v2 Avg AC (Mar 12, 2026) Aider Code Editing Leaderboard percent_correct_pct (Mar 12, 2026) |
| #47 | Phi-3-medium-128k-instruct | | 15.6% | 3 | RepoQA Official Results overall_average_pass_at_1_pct (Mar 12, 2026) RepoQA Official Results all_average_pass_at_1_pct (Mar 12, 2026) |