engineering
CAD scripting helper
Generate and debug CAD automation scripts and parametric geometry code.
#1 Recommendation
anthropic/claude-sonnet-4.6
Strong on OpenHands Issue Resolution issue_resolution_score_pct (72%) and OpenHands Index issue_resolution_score_pct (72%)
external/anthropic/claude-sonnet-4-6
22.4%
Score
36.4%
Confidence
Limited benchmark evidence for this use case.
37 ranked models with average evidence of 15.8 points. Rankings may shift as more benchmark data is ingested.
Ranked Models
30
Evidence Quality
80%
Scoring
Benchmark-backed
Top Signal
OpenHands Issue Resolution: issue_resolution_score_pct
All Ranked Models
Compare Models
Model A leads by +6.0%
Shareable Link →Model A
anthropic/claude-sonnet-4.6
external/anthropic/claude-sonnet-4-6
Rank #2
OpenHands Issue Resolution: issue_resolution_score_pct
Value 71.8% · Conf 100.0% · Weight 2.9%
openhands_issue_resolution.issue_resolution_score_pct (Mar 12, 2026)
OpenHands Index: issue_resolution_score_pct
Value 71.8% · Conf 100.0% · Weight 2.4%
openhands_index.issue_resolution_score_pct (Mar 12, 2026)
Vals SWE-bench: overall_accuracy_pct
Value 95.1% · Conf 100.0% · Weight 1.8%
vals_swebench.overall_accuracy_pct (Mar 12, 2026)
Vals LiveCodeBench: overall_accuracy_pct
Value 91.2% · Conf 100.0% · Weight 1.5%
vals_lcb.overall_accuracy_pct (Mar 12, 2026)
Model B
Kimi K2 Thinking
external/kimi/kimi-k2-thinking
Rank #10
Sonar Java Quality Leaderboard: functional_skill_pct
Value 88.4% · Conf 100.0% · Weight 2.6%
sonar_java_quality.functional_skill_pct (Mar 12, 2026)
Sonar Java Quality Leaderboard: issue_density_error_per_kloc
Value 66.6% · Conf 100.0% · Weight 1.5%
sonar_java_quality.issue_density_error_per_kloc (Mar 12, 2026)
Vals SWE-bench: overall_accuracy_pct
Value 63.5% · Conf 100.0% · Weight 1.2%
vals_swebench.overall_accuracy_pct (Mar 12, 2026)
Vals LiveCodeBench: overall_accuracy_pct
Value 65.1% · Conf 100.0% · Weight 1.1%
vals_lcb.overall_accuracy_pct (Mar 12, 2026)
▶Ranking Diagnostics & Missing Models
Source Lift
Ranked
37
Sources
8
Quality
Insufficient
Vals LiveCodeBench
vals_lcb
31 rows
1.3% avg lift
Vals SWE-bench
vals_swebench
29 rows
1.3% avg lift
Vals Legal Bench
vals_legal_bench
29 rows
0.3% avg lift
Vals Tax Eval v2
vals_tax_eval_v2
29 rows
0.3% avg lift
Missing Strong Models
gemini-2.5-flash
external/google/gemini-2-5-flash
Rank #12
17.9%
qwen-2.5-72b-instruct
external/qwen/qwen-2-5-72b-instruct
Rank #27
14.2%
x-ai/grok-3
external/x-ai/grok-3
Rank #35
12.1%
o3-20250416
external/openai/o3-20250416
Rank #43
11.7%
▶Taxonomy Details
Core Tasks
Required Modes
Domains
Related Use Cases
engineering
Component selection assistant
Recommend components under constraints with evidence and tradeoffs.
Top: gemini-3-pro-preview
engineering
Simulation setup assistant
Turn design requirements into simulation setup checklists and boundary notes.
Top: gemini-3-pro-preview
engineering
Verilog/VHDL generation
Generate RTL code and testbenches from functional specs.
Top: z-ai/glm-4.7