industrial_ops
Shift handover summary
Summarize shift notes into incidents, actions, and open items.
#1 Recommendation
gpt-4.1-20250414
Strong on MMLongBench-Doc Leaderboard acc_score_pct (75%) and Galileo Agent Leaderboard v2 Avg TSQ (64%)
external/openai/gpt-4-1-20250414
20.8%
Score
28.8%
Confidence
Limited benchmark evidence for this use case.
43 ranked models with average evidence of 13.5 points. Rankings may shift as more benchmark data is ingested.
Ranked Models
30
Evidence Quality
78%
Scoring
Benchmark-backed
Top Signal
MMLongBench-Doc Leaderboard: acc_score_pct
All Ranked Models
Compare Models
Model A leads by +4.9%
Shareable Link →Model A
gpt-4.1-20250414
external/openai/gpt-4-1-20250414
Rank #1
MMLongBench-Doc Leaderboard: acc_score_pct
Value 74.6% · Conf 100.0% · Weight 5.4%
mmlongbench_doc_leaderboard.acc_score_pct (Mar 12, 2026)
Galileo Agent Leaderboard v2: Avg TSQ
Value 64.1% · Conf 100.0% · Weight 2.3%
galileo_agent_v2.avg_tsq (Mar 12, 2026)
Galileo Agent Leaderboard v2: Avg AC
Value 100.0% · Conf 100.0% · Weight 1.9%
galileo_agent_v2.avg_ac (Mar 12, 2026)
Vectara HHEM Leaderboard: overall_hallucination_error_pct
Value 82.5% · Conf 100.0% · Weight 0.6%
vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)
Model B
gemini-2.5-pro
external/google/gemini-2-5-pro
Rank #2
Galileo Agent Leaderboard v2: Avg TSQ
Value 79.5% · Conf 100.0% · Weight 2.9%
galileo_agent_v2.avg_tsq (Mar 12, 2026)
SciArena Leaderboard: rating_elo
Value 70.7% · Conf 100.0% · Weight 1.6%
sciarena_leaderboard.rating_elo (Mar 12, 2026)
Galileo Agent Leaderboard v2: Avg AC
Value 58.7% · Conf 100.0% · Weight 1.1%
galileo_agent_v2.avg_ac (Mar 12, 2026)
Vectara HHEM Leaderboard: overall_hallucination_error_pct
Value 76.0% · Conf 100.0% · Weight 0.6%
vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)
▶Ranking Diagnostics & Missing Models
Source Lift
Ranked
43
Sources
8
Quality
Insufficient
Vals CorpFin v2
vals_corp_fin_v2
36 rows
0.4% avg lift
Vals Tax Eval v2
vals_tax_eval_v2
35 rows
0.5% avg lift
Vals Legal Bench
vals_legal_bench
34 rows
0.5% avg lift
Vals LiveCodeBench
vals_lcb
34 rows
0.4% avg lift
Missing Strong Models
alibaba/qwen3.5-flash
external/alibaba/qwen3-5-flash
Rank #33
12.3%
anthropic/claude-opus-4-1-20250805
external/anthropic/claude-opus-4-1-20250805
Rank #48
10.8%
qwen/qwen3-max
external/qwen/qwen3-max
Rank #55
10.3%
google/gemini-2.0-flash-001
external/google/gemini-2-0-flash-001
Rank #56
10.3%
▶Taxonomy Details
Core Tasks
Required Modes
Domains
Related Use Cases
industrial_ops
Operator support chat
Real-time operator assistant with grounded troubleshooting and escalation.
Top: gemini-3-pro-preview
industrial_ops
Maintenance RCA memo
Turn logs and notes into a maintenance root cause analysis.
Top: gemini-3-pro-preview
industrial_ops
Manuals Q&A (RAG grounded)
Answer operator questions grounded in technical manuals and runbooks.
Top: gemini-3-pro-preview