customer_experience
Support bot (RAG grounded)
Support chatbot grounded in docs with optional citations and escalation.
#1 Recommendation
gemini-3-pro-preview
Strong on BasedAGI Support Bot Eval overall_score_pct (86%) and FACTS Benchmark Suite facts_grounding_score_pct (88%)
external/google/gemini-3-pro-preview
36.3%
Score
48.2%
Confidence
Limited benchmark evidence for this use case.
47 ranked models with average evidence of 15.6 points. Rankings may shift as more benchmark data is ingested.
Ranked Models
30
Evidence Quality
83%
Scoring
Benchmark-backed
Top Signal
BasedAGI Support Bot Eval: overall_score_pct
All Ranked Models
Compare Models
Model A leads by +4.1%
Shareable Link →Model A
gemini-3-pro-preview
external/google/gemini-3-pro-preview
Rank #1
BasedAGI Support Bot Eval: overall_score_pct
Value 85.7% · Conf 100.0% · Weight 3.2%
basedagi_support_bot_eval.overall_score_pct (Mar 12, 2026)
FACTS Benchmark Suite: facts_grounding_score_pct
Value 88.3% · Conf 100.0% · Weight 3.0%
facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)
FACTS Benchmark Suite: facts_search_score_pct
Value 100.0% · Conf 100.0% · Weight 2.6%
facts_benchmark_suite.facts_search_score_pct (Mar 12, 2026)
Vals Finance Agent: overall_accuracy_pct
Value 87.0% · Conf 100.0% · Weight 2.6%
vals_finance_agent.overall_accuracy_pct (Mar 12, 2026)
Model B
gemini-2.5-pro
external/google/gemini-2-5-pro
Rank #2
FACTS Benchmark Suite: facts_grounding_score_pct
Value 100.0% · Conf 100.0% · Weight 3.4%
facts_benchmark_suite.facts_grounding_score_pct (Mar 12, 2026)
Vectara HHEM Leaderboard: overall_hallucination_error_pct
Value 76.0% · Conf 100.0% · Weight 2.8%
vectara_hhem_leaderboard.overall_hallucination_error_pct (Mar 12, 2026)
Vectara HHEM Leaderboard: overall_answer_rate_pct
Value 97.6% · Conf 100.0% · Weight 2.1%
vectara_hhem_leaderboard.overall_answer_rate_pct (Mar 12, 2026)
BasedAGI Support Bot Eval: overall_score_pct
Value 57.1% · Conf 100.0% · Weight 2.1%
basedagi_support_bot_eval.overall_score_pct (Mar 12, 2026)
▶Ranking Diagnostics & Missing Models
Source Lift
Ranked
47
Sources
8
Quality
Insufficient
Vals CorpFin v2
vals_corp_fin_v2
40 rows
1.2% avg lift
Vals Legal Bench
vals_legal_bench
33 rows
0.3% avg lift
Vals MedQA
vals_medqa
33 rows
0.3% avg lift
Vals Tax Eval v2
vals_tax_eval_v2
31 rows
0.3% avg lift
Missing Strong Models
gpt-4o-20241120
external/openai/gpt-4o-20241120
Rank #49
10.7%
gpt-4o-2024-05-13
external/openai/gpt-4o-2024-05-13
Rank #51
10.5%
google/gemini-2.0-flash-001
external/google/gemini-2-0-flash-001
Rank #56
10.3%
▶Taxonomy Details
Core Tasks
Required Modes
Domains
Related Use Cases
customer_experience
Agent-assist reply suggestions
Draft replies for human agents with tone and policy constraints.
Top: gemini-3-pro-preview
customer_experience
Support dialogue agent
Multi-turn support conversations with escalation and policy awareness.
Top: gemini-3-pro-preview
customer_experience
Customer feedback theme mining
Extract themes and trends from reviews, tickets, and surveys.
Top: gemini-3-pro-preview
customer_experience
Support FAQ bot
Answer common support questions with safe troubleshooting steps.
Top: gemini-2.5-pro