| #1 | gemini-3-pro-preview Strong on FACTS Benchmark Suite facts_grounding_score_pct (88%) and FACTS Benchmark Suite facts_search_score_pct (100%) | | 26.0% | 23 | FACTS Benchmark Suite facts_grounding_score_pct (Mar 16, 2026) FACTS Benchmark Suite facts_search_score_pct (Mar 16, 2026) |
| #2 | gpt-4o-20241120 Strong on DuckDB NSQL Leaderboard all_execution_accuracy (96%) and DuckDB NSQL Leaderboard hard_execution_accuracy (75%) | | 36.6% | 16 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) DuckDB NSQL Leaderboard hard_execution_accuracy (Mar 16, 2026) |
| #4 | gpt-4o | | 34.7% | 14 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) JSONSchemaBench Leaderboard medium_schema_compliance_pct (Mar 12, 2026) |
| #5 | qwen-2.5-72b-instruct | | 27.5% | 12 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) JSONSchemaBench Leaderboard medium_schema_compliance_pct (Mar 12, 2026) |
| #6 | gemini-2.5-pro | | 25.9% | 24 | FACTS Benchmark Suite facts_grounding_score_pct (Mar 16, 2026) Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) |
| #7 | deepseek/deepseek-r1 | | 30.5% | 17 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) DuckDB NSQL Leaderboard hard_execution_accuracy (Mar 16, 2026) |
| #8 | anthropic/claude-sonnet-4.6 | | 19.0% | 17 | Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #9 | gpt-5-mini-2025-08-07 | | 21.9% | 19 | Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) FACTS Benchmark Suite facts_grounding_score_pct (Mar 16, 2026) |
| #10 | gpt-5-2025-08-07 | | 18.0% | 18 | FACTS Benchmark Suite facts_grounding_score_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #13 | Grok-4-0709 | | 20.2% | 20 | Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #15 | gpt-4o-2024-08-06 | | 28.9% | 16 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) |
| #16 | google/gemini-3.1-pro-preview | | 16.1% | 18 | Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #17 | openai/gpt-5.4-2026-03-05 | | 15.8% | 17 | Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #18 | gpt-4.1-20250414 | | 20.3% | 19 | Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #21 | claude-opus-4-5-20251101 | | 18.3% | 17 | FACTS Benchmark Suite facts_grounding_score_pct (Mar 16, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #22 | openai/gpt-4o-mini-2024-07-18 | | 23.1% | 13 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) DuckDB NSQL Leaderboard hard_execution_accuracy (Mar 16, 2026) |
| #26 | gpt-5.1-2025-11-13 | | 18.0% | 18 | Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #27 | claude-sonnet-4-20250514 | | 20.1% | 18 | Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) FACTS Benchmark Suite facts_grounding_score_pct (Mar 16, 2026) |
| #29 | gemini-3-flash-preview | | 15.6% | 17 | Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #31 | google/gemini-3.1-flash-lite-preview | | 15.4% | 16 | Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #32 | xai-org/grok-4-fast-reasoning | | 15.8% | 17 | Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #34 | Qwen3-32B | | 18.4% | 4 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) |
| #35 | phi-4 | | 19.3% | 6 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) |
| #36 | gemini-2.5-flash | | 16.9% | 15 | FACTS Benchmark Suite facts_grounding_score_pct (Mar 16, 2026) Vectara HHEM Leaderboard overall_hallucination_error_pct (Mar 12, 2026) |
| #39 | xai-org/grok-4-1-fast-reasoning | | 15.8% | 17 | Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #40 | anthropic/claude-opus-4-6-thinking | | 11.5% | 15 | Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #41 | gpt-5.2-2025-12-11 | | 11.5% | 15 | Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #43 | kimi/kimi-k2.5-thinking | | 13.9% | 17 | Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) |
| #46 | anthropic/claude-opus-4-5-20251101-thinking | | 11.5% | 15 | Vals Finance Agent overall_accuracy_pct (Mar 16, 2026) Vals CorpFin v2 overall_accuracy_pct (Mar 16, 2026) |
| #47 | google/gemini-2.0-flash-001 | | 21.2% | 13 | DuckDB NSQL Leaderboard all_execution_accuracy (Mar 16, 2026) DuckDB NSQL Leaderboard hard_execution_accuracy (Mar 16, 2026) |