BasedAGIBasedAGI
Menu
Rankings live

industrial_ops

Operator support chat

Real-time operator assistant with grounded troubleshooting and escalation.

#1 Recommendation

gemini-3-pro-preview

Strong on FACTS Benchmark Suite facts_grounding_score_pct (88%) and SciArena Leaderboard rating_elo (79%)

external/google/gemini-3-pro-preview

30.7%

Score

39.9%

Confidence

Runners-up:#2 gemini-2.5-pro (28.6%)#3 anthropic/claude-sonnet-4.6 (22.7%)#4 Grok-4-0709 (22.7%)

Limited benchmark evidence for this use case.

50 ranked models with average evidence of 14.6 points. Rankings may shift as more benchmark data is ingested.

Ranked Models

30

Evidence Quality

83%

Scoring

Benchmark-backed

Top Signal

FACTS Benchmark Suite: facts_grounding_score_pct

All Ranked Models

Max params:
Min confidence:
30 of 30
RankModelScore
#1gemini-3-pro-preview

Strong on FACTS Benchmark Suite facts_grounding_score_pct (88%) and SciArena Leaderboard rating_elo (79%)

30.7%
#2gemini-2.5-pro

Strong on FACTS Benchmark Suite facts_grounding_score_pct (100%) and Vectara HHEM Leaderboard overall_hallucination_error_pct (76%)

28.6%
#3anthropic/claude-sonnet-4.6

Strong on Vals Finance Agent overall_accuracy_pct (100%) and Vals CorpFin v2 overall_accuracy_pct (91%)

22.7%
#4Grok-4-0709
22.7%
#5gpt-4.1-20250414
22.5%
#6gpt-5-mini-2025-08-07
21.5%
#7gpt-5-2025-08-07
21.2%
#8google/gemini-3.1-pro-preview
20.2%
#9openai/gpt-5.4-2026-03-05
19.8%
#10claude-sonnet-4-20250514
19.6%
#11claude-opus-4-5-20251101
19.2%
#12gemini-2.5-flash
18.0%
#13gpt-5.1-2025-11-13
17.5%
#14gemini-3-flash-preview
17.1%
#15google/gemini-3.1-flash-lite-preview
16.4%
#16xai-org/grok-4-fast-reasoning
16.2%
#17xai-org/grok-4-1-fast-reasoning
15.4%
#18anthropic/claude-opus-4-6-thinking
15.4%
#19gpt-5.2-2025-12-11
15.3%
#20kimi/kimi-k2.5-thinking
14.8%
#21anthropic/claude-opus-4-5-20251101-thinking
14.2%
#22x-ai/grok-3
13.7%
#23zai/glm-5-thinking
13.4%
#24anthropic/claude-sonnet-4-5-20250929-thinking
13.3%
#26qwen-2.5-72b-instruct
12.5%
#27o3-20250416
12.1%
#28mistralai/mistral-large-2512
12.0%
#29anthropic/claude-haiku-4-5-20251001-thinking
11.8%
#30alibaba/qwen3.5-flash
11.7%
#31xai-org/grok-4-1-fast-non-reasoning
11.6%

Compare Models

Model A leads by +2.1%

Shareable Link →

Model A

gemini-3-pro-preview

external/google/gemini-3-pro-preview

30.7%

Rank #1

Confidence 39.9%23 evidence pts

FACTS Benchmark Suite: facts_grounding_score_pct

Value 88.3% · Conf 100.0% · Weight 2.8%

facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)

SciArena Leaderboard: rating_elo

Value 78.8% · Conf 100.0% · Weight 2.6%

sciarena_leaderboard.rating_elo (Mar 12, 2026)

FACTS Benchmark Suite: facts_search_score_pct

Value 100.0% · Conf 100.0% · Weight 2.4%

facts_benchmark_suite.facts_search_score_pct (Mar 12, 2026)

FACTS Benchmark Suite: average_score_pct

Value 100.0% · Conf 100.0% · Weight 2.3%

facts_benchmark_suite.average_score_pct (Mar 12, 2026)

Model B

gemini-2.5-pro

external/google/gemini-2-5-pro

28.6%

Rank #2

Confidence 41.2%23 evidence pts

FACTS Benchmark Suite: facts_grounding_score_pct

Value 100.0% · Conf 100.0% · Weight 3.2%

facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)

Vectara HHEM Leaderboard: overall_hallucination_error_pct

Value 76.0% · Conf 100.0% · Weight 2.5%

vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)

SciArena Leaderboard: rating_elo

Value 70.7% · Conf 100.0% · Weight 2.3%

sciarena_leaderboard.rating_elo (Mar 12, 2026)

Galileo Agent Leaderboard v2: Avg TSQ

Value 79.5% · Conf 100.0% · Weight 2.3%

galileo_agent_v2.avg_tsq (Mar 12, 2026)

▶Ranking Diagnostics & Missing Models

Source Lift

Ranked

50

Sources

8

Quality

Insufficient

Vals CorpFin v2

vals_corp_fin_v2

42 rows

1.6% avg lift

Vals Legal Bench

vals_legal_bench

38 rows

0.4% avg lift

Vals Tax Eval v2

vals_tax_eval_v2

37 rows

0.4% avg lift

Vals MedQA

vals_medqa

36 rows

0.4% avg lift

Missing Strong Models

No obvious gaps right now.

▶Taxonomy Details

Core Tasks

task.customer_support_dialoguetask.rag_answer_no_citations

Required Modes

none

Domains

domain.industrial_ops_maintenance

Related Use Cases

industrial_ops

Maintenance RCA memo

Turn logs and notes into a maintenance root cause analysis.

Top: gemini-3-pro-preview

industrial_ops

Manuals Q&A (RAG grounded)

Answer operator questions grounded in technical manuals and runbooks.

Top: gemini-3-pro-preview

industrial_ops

Shift handover summary

Summarize shift notes into incidents, actions, and open items.

Top: gpt-4.1-20250414