devops_sre
Agentic incident response
Agentic tool-using workflow for incident triage and remediation planning.
#1 Recommendation
gemini-2.5-pro
Strong on Galileo Agent Leaderboard v2 Avg AC (59%) and FACTS Benchmark Suite facts_grounding_score_pct (100%)
external/google/gemini-2-5-pro
23.4%
Score
34.3%
Confidence
Limited benchmark evidence for this use case.
45 ranked models with average evidence of 15.9 points. Rankings may shift as more benchmark data is ingested.
Ranked Models
30
Evidence Quality
81%
Scoring
Benchmark-backed
Top Signal
Galileo Agent Leaderboard v2: Avg AC
All Ranked Models
Compare Models
Model A leads by +0.2%
Shareable Link →Model A
gemini-2.5-pro
external/google/gemini-2-5-pro
Rank #1
Galileo Agent Leaderboard v2: Avg AC
Value 58.7% · Conf 100.0% · Weight 2.8%
galileo_agent_v2.avg_ac (Mar 12, 2026)
FACTS Benchmark Suite: facts_grounding_score_pct
Value 100.0% · Conf 100.0% · Weight 2.5%
facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)
Vectara HHEM Leaderboard: overall_hallucination_error_pct
Value 76.0% · Conf 100.0% · Weight 2.0%
vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)
Galileo Agent Leaderboard v2: Avg TSQ
Value 79.5% · Conf 100.0% · Weight 1.6%
galileo_agent_v2.avg_tsq (Mar 12, 2026)
Model B
gpt-4.1-20250414
external/openai/gpt-4-1-20250414
Rank #2
Galileo Agent Leaderboard v2: Avg AC
Value 100.0% · Conf 100.0% · Weight 4.8%
galileo_agent_v2.avg_ac (Mar 12, 2026)
MMLongBench-Doc Leaderboard: acc_score_pct
Value 74.6% · Conf 100.0% · Weight 2.5%
mmlongbench_doc_leaderboard.acc_score_pct (Mar 12, 2026)
Vectara HHEM Leaderboard: overall_hallucination_error_pct
Value 82.5% · Conf 100.0% · Weight 2.2%
vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)
Vals CorpFin v2: overall_accuracy_pct
Value 84.9% · Conf 100.0% · Weight 1.7%
vals_corp_fin_v2.overall_accuracy_pct (Mar 12, 2026)
▶Ranking Diagnostics & Missing Models
Source Lift
Ranked
45
Sources
8
Quality
Insufficient
Vals CorpFin v2
vals_corp_fin_v2
41 rows
1.3% avg lift
Vals Legal Bench
vals_legal_bench
37 rows
0.3% avg lift
Vals Tax Eval v2
vals_tax_eval_v2
35 rows
0.3% avg lift
Vals MedQA
vals_medqa
34 rows
0.3% avg lift
Missing Strong Models
gpt-4o
external/openai/gpt-4o
Rank #22
15.2%
gpt-4o-2024-05-13
external/openai/gpt-4o-2024-05-13
Rank #51
10.5%
deepseek/deepseek-r1
external/deepseek/deepseek-r1
Rank #54
10.5%
google/gemini-2.0-flash-001
external/google/gemini-2-0-flash-001
Rank #56
10.3%
▶Taxonomy Details
Core Tasks
Required Modes
Domains
Related Use Cases
devops_sre
Runbook step assistant
Suggest safe runbook steps and escalation points grounded in docs.
Top: gemini-3-pro-preview
devops_sre
Log triage
Interpret logs and propose safe diagnostic steps.
Top: gemini-3-pro-preview
devops_sre
Config debugging
Diagnose and patch YAML/JSON/TOML configs with minimal diffs.
Top: gpt-4.1-20250414
devops_sre
Kubernetes manifest generation
Generate K8s manifests with safe defaults and probes.
Top: gpt-4.1-20250414