developer_tools
Integration test generation
Generate integration tests with setup/teardown and stable fixtures.
#1 Recommendation
deepseek/deepseek-r1
Strong on Aider Polyglot Leaderboard percent_correct_pct (80%) and Sonar Java Quality Leaderboard functional_skill_pct (83%)
external/deepseek/deepseek-r1
18.8%
Score
26.4%
Confidence
Limited benchmark evidence for this use case.
49 ranked models with average evidence of 13.3 points. Rankings may shift as more benchmark data is ingested.
Ranked Models
30
Evidence Quality
80%
Scoring
Benchmark-backed
Top Signal
Aider Polyglot Leaderboard: percent_correct_pct
All Ranked Models
Compare Models
Model A leads by +0.9%
Shareable Link →Model A
deepseek/deepseek-r1
external/deepseek/deepseek-r1
Rank #2
Aider Polyglot Leaderboard: percent_correct_pct
Value 80.0% · Conf 100.0% · Weight 2.7%
aider_polyglot.percent_correct_pct (Mar 12, 2026)
Sonar Java Quality Leaderboard: functional_skill_pct
Value 82.8% · Conf 100.0% · Weight 2.2%
sonar_java_quality.functional_skill_pct (Mar 12, 2026)
Aider Polyglot Leaderboard: correct_edit_format_pct
Value 90.4% · Conf 100.0% · Weight 1.3%
aider_polyglot.correct_edit_format_pct (Mar 12, 2026)
Sonar Java Quality Leaderboard: issue_density_error_per_kloc
Value 59.0% · Conf 100.0% · Weight 0.9%
sonar_java_quality.issue_density_error_per_kloc (Mar 12, 2026)
Model B
z-ai/glm-4.7
external/z-ai/glm-4-7
Rank #3
Sonar Java Quality Leaderboard: functional_skill_pct
Value 74.4% · Conf 100.0% · Weight 2.0%
sonar_java_quality.functional_skill_pct (Mar 12, 2026)
Vals LiveCodeBench: overall_accuracy_pct
Value 91.4% · Conf 100.0% · Weight 1.3%
vals_lcb.overall_accuracy_pct (Mar 12, 2026)
Vals SWE-bench: overall_accuracy_pct
Value 79.9% · Conf 100.0% · Weight 1.3%
vals_swebench.overall_accuracy_pct (Mar 12, 2026)
Sonar Java Quality Leaderboard: issue_density_error_per_kloc
Value 65.2% · Conf 100.0% · Weight 1.0%
sonar_java_quality.issue_density_error_per_kloc (Mar 12, 2026)
▶Ranking Diagnostics & Missing Models
Source Lift
Ranked
49
Sources
8
Quality
Insufficient
Vals LiveCodeBench
vals_lcb
38 rows
1.1% avg lift
Vals Legal Bench
vals_legal_bench
38 rows
0.2% avg lift
Vals Tax Eval v2
vals_tax_eval_v2
38 rows
0.2% avg lift
Vals CorpFin v2
vals_corp_fin_v2
37 rows
0.2% avg lift
Missing Strong Models
anthropic/claude-opus-4-1-20250805
external/anthropic/claude-opus-4-1-20250805
Rank #48
10.8%
google/gemini-2.0-flash-001
external/google/gemini-2-0-flash-001
Rank #56
10.3%
deepseek-v3
external/deepseek-ai/deepseek-v3
Rank #66
8.8%
Llama-4-Scout-17B-16E-Instruct
external/meta/llama-4-scout-17b-16e-instruct
Rank #73
8.4%
▶Taxonomy Details
Core Tasks
Required Modes
Domains
Related Use Cases
developer_tools
Codebase onboarding brief
Summarize a repository's architecture, modules, and conventions.
Top: gemini-3-pro-preview
developer_tools
Code generation
Generate correct, secure code from requirements.
Top: anthropic/claude-sonnet-4.6
developer_tools
PR review agent
Review diffs for correctness, security, and maintainability.
Top: anthropic/claude-sonnet-4.6
developer_tools
Agentic bug fixing
Agentic loop that reproduces, fixes, and validates with tests.
Top: anthropic/claude-sonnet-4.6