BasedAGIBasedAGI
Menu
Rankings live

devops_sre

Log triage

Interpret logs and propose safe diagnostic steps.

#1 Recommendation

gemini-3-pro-preview

Strong on BasedAGI Log Triage Eval overall_score_pct (87%) and FACTS Benchmark Suite facts_grounding_score_pct (88%)

external/google/gemini-3-pro-preview

42.5%

Score

53.5%

Confidence

Runners-up:#2 gemini-2.5-pro (36.8%)#3 anthropic/claude-sonnet-4.6 (33.1%)#4 Grok-4-0709 (28.9%)

Limited benchmark evidence for this use case.

47 ranked models with average evidence of 15.3 points. Rankings may shift as more benchmark data is ingested.

Ranked Models

30

Evidence Quality

84%

Scoring

Benchmark-backed

Top Signal

BasedAGI Log Triage Eval: overall_score_pct

All Ranked Models

Max params:
Min confidence:
30 of 30
RankModelScore
#1gemini-3-pro-preview

Strong on BasedAGI Log Triage Eval overall_score_pct (87%) and FACTS Benchmark Suite facts_grounding_score_pct (88%)

42.5%
#2gemini-2.5-pro

Strong on BasedAGI Log Triage Eval overall_score_pct (60%) and FACTS Benchmark Suite facts_grounding_score_pct (100%)

36.8%
#3anthropic/claude-sonnet-4.6

Strong on BasedAGI Log Triage Eval overall_score_pct (73%) and Vals Finance Agent overall_accuracy_pct (100%)

33.1%
#4Grok-4-0709
28.9%
#5gpt-4.1-20250414
23.3%
#6gpt-5-mini-2025-08-07
22.1%
#7gpt-5-2025-08-07
21.7%
#8google/gemini-3.1-pro-preview
20.7%
#9claude-sonnet-4-20250514
20.5%
#10openai/gpt-5.4-2026-03-05
20.3%
#11claude-opus-4-5-20251101
19.7%
#13gpt-5.1-2025-11-13
18.0%
#14kimi/kimi-k2.5-thinking
17.6%
#15gemini-3-flash-preview
17.5%
#16gemini-2.5-flash
16.9%
#17google/gemini-3.1-flash-lite-preview
16.9%
#18xai-org/grok-4-fast-reasoning
16.6%
#19xai-org/grok-4-1-fast-reasoning
15.8%
#20anthropic/claude-opus-4-6-thinking
15.8%
#21gpt-5.2-2025-12-11
15.7%
#22anthropic/claude-opus-4-5-20251101-thinking
14.6%
#23anthropic/claude-sonnet-4-5-20250929-thinking
13.7%
#25zai/glm-5-thinking
13.3%
#26x-ai/grok-3
13.2%
#27o3-20250416
12.4%
#28mistralai/mistral-large-2512
12.3%
#30anthropic/claude-haiku-4-5-20251001-thinking
12.1%
#31alibaba/qwen3.5-flash
12.0%
#32xai-org/grok-4-1-fast-non-reasoning
11.9%
#33gpt-4.1-mini-20250414
11.7%

Compare Models

Model A leads by +5.8%

Shareable Link →

Model A

gemini-3-pro-preview

external/google/gemini-3-pro-preview

42.5%

Rank #1

Confidence 53.5%27 evidence pts

BasedAGI Log Triage Eval: overall_score_pct

Value 86.7% · Conf 100.0% · Weight 4.7%

basedagi_log_triage_eval.overall_score_pct (Mar 12, 2026)

FACTS Benchmark Suite: facts_grounding_score_pct

Value 88.3% · Conf 100.0% · Weight 2.8%

facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)

FACTS Benchmark Suite: facts_search_score_pct

Value 100.0% · Conf 100.0% · Weight 2.4%

facts_benchmark_suite.facts_search_score_pct (Mar 12, 2026)

FACTS Benchmark Suite: average_score_pct

Value 100.0% · Conf 100.0% · Weight 2.2%

facts_benchmark_suite.average_score_pct (Mar 12, 2026)

Model B

gemini-2.5-pro

external/google/gemini-2-5-pro

36.8%

Rank #2

Confidence 55.6%27 evidence pts

BasedAGI Log Triage Eval: overall_score_pct

Value 60.0% · Conf 100.0% · Weight 3.2%

basedagi_log_triage_eval.overall_score_pct (Mar 12, 2026)

FACTS Benchmark Suite: facts_grounding_score_pct

Value 100.0% · Conf 100.0% · Weight 3.1%

facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)

Vectara HHEM Leaderboard: overall_hallucination_error_pct

Value 76.0% · Conf 100.0% · Weight 2.5%

vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)

Galileo Agent Leaderboard v2: Avg AC

Value 58.7% · Conf 100.0% · Weight 2.3%

galileo_agent_v2.avg_ac (Mar 12, 2026)

▶Ranking Diagnostics & Missing Models

Source Lift

Ranked

47

Sources

8

Quality

Insufficient

Vals CorpFin v2

vals_corp_fin_v2

42 rows

1.5% avg lift

Vals Legal Bench

vals_legal_bench

37 rows

0.4% avg lift

Vals Tax Eval v2

vals_tax_eval_v2

36 rows

0.4% avg lift

Vals MedQA

vals_medqa

34 rows

0.4% avg lift

Missing Strong Models

gpt-4o

external/openai/gpt-4o

Rank #22

15.2%

Thin evidence after weighting

deepseek/deepseek-r1

external/deepseek/deepseek-r1

Rank #54

10.5%

Thin evidence after weighting
▶Taxonomy Details

Core Tasks

task.log_triagetask.risk_assessment

Required Modes

none

Domains

domain.devops_sre

Related Use Cases

devops_sre

Runbook step assistant

Suggest safe runbook steps and escalation points grounded in docs.

Top: gemini-3-pro-preview

devops_sre

Config debugging

Diagnose and patch YAML/JSON/TOML configs with minimal diffs.

Top: gpt-4.1-20250414

devops_sre

Kubernetes manifest generation

Generate K8s manifests with safe defaults and probes.

Top: gpt-4.1-20250414

devops_sre

Terraform generation

Generate Terraform IaC with correct resources and safe defaults.

Top: gpt-4.1-20250414