DevOps & SRE

RCA writeup

Draft root cause analysis documents with concrete prevention actions.

task.root_cause_analysis_writeuptask.risk_assessment

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

claude-sonnet-4

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

28.9%

Best benchmark score

39.2%

Confidence

All ranked models — top 3

🥇

claude-sonnet-4

28.9%

🥈

gpt-4.1-20250414

28.4%

🥉

gemini-2.5-pro

25.0%

Ranked Models

Evidence Quality

81%

Evidence Points

Top Signal

Galileo Agent Leaderboard v2: Avg AC

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	claude-sonnet-4 Strong on Galileo Agent Leaderboard v2 Avg AC and SWE-bench Verified Leaderboard swe_verified_resolved_pct	28.9%	39%	$6.00	Galileo Agent Leaderboard v2SWE-bench Verified Leaderboard
🥈	gpt-4.1-20250414 Strong on Galileo Agent Leaderboard v2 Avg AC and MMLongBench-Doc Leaderboard acc_score_pct	28.4%	41%	—	Galileo Agent Leaderboard v2MMLongBench-Doc Leaderboard
🥉	gemini-2.5-pro Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Galileo Agent Leaderboard v2 Avg AC	25.0%	36%	$3.44	SWE-bench Verified LeaderboardGalileo Agent Leaderboard v2
#4	gpt-5-2025-08-07 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	22.9%	28%	—	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#5	gemini-3-pro-preview Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and BFCL Multi-turn Official Multi Turn Acc	20.4%	26%	$4.50	SWE-bench Verified LeaderboardBFCL Multi-turn Official
#6	gpt-5.2-2025-12-11 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and BFCL Multi-turn Official Multi Turn Acc	19.9%	24%	—	SWE-bench Verified LeaderboardBFCL Multi-turn Official
#7	o3-20250416 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and OSWorld Official Leaderboard overall_score_pct	19.8%	31%	$3.50	SWE-bench Verified LeaderboardOSWorld Official Leaderboard
#8	Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg AC and Galileo Agent Leaderboard v2 Avg TSQ	18.8%	29%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#9	gpt-5-mini-2025-08-07 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals MedQA overall_accuracy_pct	18.6%	27%	—	SWE-bench Verified LeaderboardVals MedQA
#10	claude-opus-4-5-20251101 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals Mortgage Tax overall_accuracy_pct	17.6%	23%	—	SWE-bench Verified LeaderboardVals Mortgage Tax
#11	gemini-3.1-pro-preview Strong on Vals MedCode overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct	15.8%	18%	$4.50	Vals MedCodeVals LiveCodeBench
#12	gpt-4.1-mini-20250414 Strong on Galileo Agent Leaderboard v2 Avg AC and SWE-bench Verified Leaderboard swe_verified_resolved_pct	15.2%	23%	—	Galileo Agent Leaderboard v2SWE-bench Verified Leaderboard
#13	Kimi-K2-Instruct Strong on Galileo Agent Leaderboard v2 Avg AC and SWE-bench Verified Leaderboard swe_verified_resolved_pct	14.9%	21%	—	Galileo Agent Leaderboard v2SWE-bench Verified Leaderboard
#14	kimi-k2.5-thinking Strong on OSWorld Official Leaderboard overall_score_pct and Vals CorpFin v2 overall_accuracy_pct	14.5%	18%	—	OSWorld Official LeaderboardVals CorpFin v2
#15	gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Avg AC and Galileo Agent Leaderboard v2 Avg TSQ	14.0%	24%	$0.17	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#16	o4-mini Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	13.9%	25%	$1.93	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#17	Kimi K2 Thinking Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals MedQA overall_accuracy_pct	13.2%	18%	$1.07	SWE-bench Verified LeaderboardVals MedQA
#18	gpt-4o-20241120 Strong on MMLongBench-Doc Leaderboard acc_score_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct	13.2%	25%	—	MMLongBench-Doc LeaderboardSWE-bench Verified Leaderboard
#19	gemini-3-flash-preview Strong on Vals Legal Bench overall_accuracy_pct and Vals MedQA overall_accuracy_pct	12.8%	16%	$1.13	Vals Legal BenchVals MedQA
#20	claude-opus-4 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	12.8%	16%	$10.00	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#21	gpt-4.1 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and DuckDB NSQL Leaderboard all_execution_accuracy	12.1%	15%	$3.50	SWE-bench Verified LeaderboardDuckDB NSQL Leaderboard
#22	gpt-5.4-2026-03-05 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals MedQA overall_accuracy_pct	12.1%	14%	—	Vectara HHEM LeaderboardVals MedQA
#23	gpt-5.1-2025-11-13 Strong on Vals Case Law v2 overall_accuracy_pct and Vals MedScribe overall_accuracy_pct	11.7%	15%	—	Vals Case Law v2Vals MedScribe
#24	claude-sonnet-4.6 Strong on Vals Finance Agent overall_accuracy_pct and Vals Tax Eval v2 overall_accuracy_pct	11.6%	14%	$6.00	Vals Finance AgentVals Tax Eval v2
#25	gpt-4o Strong on MEGA-Bench app:Planning and SWE-bench Verified Leaderboard swe_verified_resolved_pct	11.6%	19%	$0.26	MEGA-BenchSWE-bench Verified Leaderboard
#28	qwen-2.5-72b-instruct Strong on Galileo Agent Leaderboard v2 Avg AC and Galileo Agent Leaderboard v2 Avg TSQ	11.1%	16%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#29	grok-4-1-fast-reasoning Strong on BFCL Multi-turn Official Multi Turn Acc and Vals CorpFin v2 overall_accuracy_pct	11.1%	17%	$0.28	BFCL Multi-turn OfficialVals CorpFin v2
#30	claude-opus-4-6-thinking Strong on Vals SWE-bench overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	11.0%	12%	—	Vals SWE-benchVals Mortgage Tax
#31	claude-opus-4-6 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and AgentSet LLM Leaderboard elo_score	10.9%	13%	$10.00	SWE-bench Verified LeaderboardAgentSet LLM Leaderboard
#32	claude-opus-4-5-20251101-thinking Strong on Vals MedQA overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	10.7%	12%	—	Vals MedQAVals Mortgage Tax

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals Legal Bench

42 rows · 0.5% avg lift

Vals GPQA

41 rows · 0.4% avg lift

Vals Tax Eval v2

40 rows · 0.5% avg lift

Vals LiveCodeBench

40 rows · 0.5% avg lift

Missing frontier models

No obvious gaps right now.

▶Taxonomy & task details

Core tasks

task.root_cause_analysis_writeuptask.risk_assessment

Required modes

mode.long_context

Domains

domain.devops_sre

Related in DevOps & SRE

Runbook step assistant

Suggest safe runbook steps and escalation points grounded in docs.

Log triage

Interpret logs and propose safe diagnostic steps.

Config debugging

Diagnose and patch YAML/JSON/TOML configs with minimal diffs.

Kubernetes manifest generation

Generate K8s manifests with safe defaults and probes.