developer_tools
Code generation
Generate correct, secure code from requirements.
#1 Recommendation
anthropic/claude-sonnet-4.6
Strong on OpenHands Issue Resolution issue_resolution_score_pct (72%) and OpenHands Index issue_resolution_score_pct (72%)
external/anthropic/claude-sonnet-4-6
19.7%
Score
33.8%
Confidence
Limited benchmark evidence for this use case.
18 ranked models with average evidence of 17.8 points. Rankings may shift as more benchmark data is ingested.
Ranked Models
18
Evidence Quality
82%
Scoring
Benchmark-backed
Top Signal
OpenHands Issue Resolution: issue_resolution_score_pct
All Ranked Models
| Rank | Model | Score |
|---|---|---|
| #7 | anthropic/claude-sonnet-4.6 | 19.7% |
| #10 | Kimi K2 Thinking | 16.3% |
| #11 | minimax/minimax-m2.1 | 15.8% |
| #12 | kimi/kimi-k2.5-thinking | 14.1% |
| #13 | deepseek/deepseek-r1 | 14.1% |
| #14 | z-ai/glm-4.7 | 13.6% |
| #17 | gemini-3-pro-preview | 11.3% |
| #18 | GLM-5 | 11.1% |
| #24 | gpt-4.1-20250414 | 9.7% |
| #25 | gpt-4o-2024-08-06 | 9.3% |
| #26 | gpt-4o | 9.1% |
| #29 | Grok-4-0709 | 9.0% |
| #31 | gpt-4o-2024-05-13 | 8.9% |
| #35 | claude-sonnet-4-20250514 | 8.7% |
| #43 | gpt-4o-20241120 | 8.2% |
| #50 | GLM-4.7 | 7.6% |
| #58 | gemini-2.5-pro | 6.7% |
| #76 | openai/gpt-4o-mini-2024-07-18 | 3.1% |
Compare Models
Model A leads by +3.4%
Shareable Link →Model A
anthropic/claude-sonnet-4.6
external/anthropic/claude-sonnet-4-6
Rank #7
OpenHands Issue Resolution: issue_resolution_score_pct
Value 71.8% · Conf 100.0% · Weight 2.4%
openhands_issue_resolution.issue_resolution_score_pct (Mar 12, 2026)
OpenHands Index: issue_resolution_score_pct
Value 71.8% · Conf 100.0% · Weight 2.0%
openhands_index.issue_resolution_score_pct (Mar 12, 2026)
OpenHands Index: greenfield_score_pct
Value 75.2% · Conf 100.0% · Weight 1.4%
openhands_index.greenfield_score_pct (Mar 12, 2026)
Vals SWE-bench: overall_accuracy_pct
Value 95.1% · Conf 100.0% · Weight 1.3%
vals_swebench.overall_accuracy_pct (Mar 12, 2026)
Model B
Kimi K2 Thinking
external/kimi/kimi-k2-thinking
Rank #10
Sonar Java Quality Leaderboard: functional_skill_pct
Value 88.4% · Conf 100.0% · Weight 2.8%
sonar_java_quality.functional_skill_pct (Mar 12, 2026)
Sonar Java Quality Leaderboard: issue_density_error_per_kloc
Value 66.6% · Conf 100.0% · Weight 1.5%
sonar_java_quality.issue_density_error_per_kloc (Mar 12, 2026)
Sonar Java Quality Leaderboard: vulnerability_density_error_per_kloc
Value 61.4% · Conf 100.0% · Weight 1.0%
sonar_java_quality.vulnerability_density_error_per_kloc (Mar 12, 2026)
Vals SWE-bench: overall_accuracy_pct
Value 63.5% · Conf 100.0% · Weight 0.9%
vals_swebench.overall_accuracy_pct (Mar 12, 2026)
▶Ranking Diagnostics & Missing Models
Source Lift
Ranked
18
Sources
8
Quality
Insufficient
Vals LiveCodeBench
vals_lcb
11 rows
0.9% avg lift
Vals SWE-bench
vals_swebench
9 rows
1.0% avg lift
Vals Terminal-Bench 2
vals_terminal_bench_2
9 rows
0.7% avg lift
Vals Legal Bench
vals_legal_bench
8 rows
0.2% avg lift
Missing Strong Models
gpt-5-mini-2025-08-07
external/openai/gpt-5-mini-2025-08-07
Rank #7
19.6%
google/gemini-3.1-pro-preview
external/google/gemini-3-1-pro-preview
Rank #8
19.3%
gpt-5-2025-08-07
external/openai/gpt-5-2025-08-07
Rank #9
19.2%
openai/gpt-5.4-2026-03-05
external/openai/gpt-5-4-2026-03-05
Rank #10
18.9%
▶Taxonomy Details
Core Tasks
Required Modes
Domains
Related Use Cases
developer_tools
Codebase onboarding brief
Summarize a repository's architecture, modules, and conventions.
Top: gemini-3-pro-preview
developer_tools
PR review agent
Review diffs for correctness, security, and maintainability.
Top: anthropic/claude-sonnet-4.6
developer_tools
Agentic bug fixing
Agentic loop that reproduces, fixes, and validates with tests.
Top: anthropic/claude-sonnet-4.6
developer_tools
Refactoring assistant
Refactor code safely while preserving behavior and improving clarity.
Top: z-ai/glm-4.7