BasedAGIBasedAGI
Menu
Rankings live

supply_chain

Vendor contract summary (procurement)

Summarize vendor contracts into key terms, risks, and deviations.

#1 Recommendation

gemini-3-pro-preview

Strong on FACTS Benchmark Suite facts_grounding_score_pct (88%) and FACTS Benchmark Suite facts_search_score_pct (100%)

external/google/gemini-3-pro-preview

29.0%

Score

37.8%

Confidence

Runners-up:#2 gemini-2.5-pro (27.7%)#3 anthropic/claude-sonnet-4.6 (19.3%)#4 gpt-4.1-20250414 (19.2%)

Limited benchmark evidence for this use case.

49 ranked models with average evidence of 14.9 points. Rankings may shift as more benchmark data is ingested.

Ranked Models

30

Evidence Quality

81%

Scoring

Benchmark-backed

Top Signal

FACTS Benchmark Suite: facts_grounding_score_pct

All Ranked Models

Max params:
Min confidence:
30 of 30
RankModelScore
#1gemini-3-pro-preview

Strong on FACTS Benchmark Suite facts_grounding_score_pct (88%) and FACTS Benchmark Suite facts_search_score_pct (100%)

29.0%
#2gemini-2.5-pro

Strong on FACTS Benchmark Suite facts_grounding_score_pct (100%) and Vectara HHEM Leaderboard overall_hallucination_error_pct (76%)

27.7%
#3anthropic/claude-sonnet-4.6

Strong on Vals Finance Agent overall_accuracy_pct (100%) and Vals CorpFin v2 overall_accuracy_pct (91%)

19.3%
#4gpt-4.1-20250414
19.2%
#5Grok-4-0709
19.0%
#6gpt-5-mini-2025-08-07
18.3%
#7gpt-5-2025-08-07
18.0%
#8google/gemini-3.1-pro-preview
17.2%
#9openai/gpt-5.4-2026-03-05
16.8%
#10claude-sonnet-4-20250514
16.4%
#11claude-opus-4-5-20251101
16.3%
#12gemini-2.5-flash
16.2%
#13gpt-5.1-2025-11-13
15.0%
#14gemini-3-flash-preview
14.6%
#15google/gemini-3.1-flash-lite-preview
14.0%
#16xai-org/grok-4-fast-reasoning
13.8%
#17xai-org/grok-4-1-fast-reasoning
13.1%
#18anthropic/claude-opus-4-6-thinking
13.1%
#19gpt-5.2-2025-12-11
13.1%
#20kimi/kimi-k2.5-thinking
12.6%
#21anthropic/claude-opus-4-5-20251101-thinking
12.1%
#22gpt-4o
11.6%
#23x-ai/grok-3
11.5%
#24zai/glm-5-thinking
11.5%
#25anthropic/claude-sonnet-4-5-20250929-thinking
11.4%
#26deepseek/deepseek-r1
11.3%
#28qwen-2.5-72b-instruct
10.5%
#29o3-20250416
10.3%
#30mistralai/mistral-large-2512
10.2%
#32anthropic/claude-haiku-4-5-20251001-thinking
10.0%

Compare Models

Model A leads by +1.3%

Shareable Link →

Model A

gemini-3-pro-preview

external/google/gemini-3-pro-preview

29.0%

Rank #1

Confidence 37.8%24 evidence pts

FACTS Benchmark Suite: facts_grounding_score_pct

Value 88.3% · Conf 100.0% · Weight 2.3%

facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)

FACTS Benchmark Suite: facts_search_score_pct

Value 100.0% · Conf 100.0% · Weight 2.0%

facts_benchmark_suite.facts_search_score_pct (Mar 12, 2026)

SciArena Leaderboard: rating_elo

Value 78.8% · Conf 100.0% · Weight 1.9%

sciarena_leaderboard.rating_elo (Mar 12, 2026)

FACTS Benchmark Suite: average_score_pct

Value 100.0% · Conf 100.0% · Weight 1.8%

facts_benchmark_suite.average_score_pct (Mar 12, 2026)

Model B

gemini-2.5-pro

external/google/gemini-2-5-pro

27.7%

Rank #2

Confidence 39.8%24 evidence pts

FACTS Benchmark Suite: facts_grounding_score_pct

Value 100.0% · Conf 100.0% · Weight 2.6%

facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)

Vectara HHEM Leaderboard: overall_hallucination_error_pct

Value 76.0% · Conf 100.0% · Weight 2.1%

vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)

LEXam Leaderboard: average_score_pct

Value 89.4% · Conf 100.0% · Weight 2.0%

lexam_leaderboard.average_score_pct (Mar 12, 2026)

SciArena Leaderboard: rating_elo

Value 70.7% · Conf 100.0% · Weight 1.7%

sciarena_leaderboard.rating_elo (Mar 12, 2026)

▶Ranking Diagnostics & Missing Models

Source Lift

Ranked

49

Sources

8

Quality

Insufficient

Vals CorpFin v2

vals_corp_fin_v2

42 rows

1.3% avg lift

Vals Legal Bench

vals_legal_bench

37 rows

0.3% avg lift

Vals Tax Eval v2

vals_tax_eval_v2

37 rows

0.3% avg lift

Vals MedQA

vals_medqa

36 rows

0.3% avg lift

Missing Strong Models

deepseek-v3

external/deepseek-ai/deepseek-v3

Rank #66

8.8%

Thin evidence after weighting
▶Taxonomy Details

Core Tasks

task.summarize_legal_contracttask.contract_term_extraction

Required Modes

mode.long_contextmode.citationsmode.json_schema

Domains

domain.procurement_vendor_mgmt

Related Use Cases

supply_chain

Disruption monitoring brief

Summarize disruptions into risk, options, and recommendations.

Top: gemini-3-pro-preview

supply_chain

Supplier risk monitoring

Track supplier risk signals from multi-source text and summarize actions.

Top: gemini-3-pro-preview

supply_chain

Route alternatives planning

Propose alternative routes and tradeoffs under disruptions.

Top: gemini-2.5-pro

supply_chain

Tail spend categorization

Categorize tail spend purchases into taxonomy buckets for sourcing.

Top: gpt-4.1-20250414