benchmark evidence
BigCodeBench Instruct
BigCodeBench bigcodebench-instruct — practical Python coding benchmark.
winner on BigCodeBench Instruct
direct benchmark result, not a broad vertical composite | source row dated 2000-01-01
scored on 2000-01-01 · stale source data (9646d)
latest mapped results | top 20
| # | Model | Score | Evidence | Tested |
|---|---|---|---|---|
| 1 | Meta: Llama 4 Maverick | 49.7 | model-only independent_benchmark | 2000-01-01 |
| 2 | Qwen2.5 Coder 32B Instruct | 49.0 | model-only independent_benchmark | 2000-01-01 |
| 3 | Meta: Llama 3.3 70B Instruct | 46.9 | model-only independent_benchmark | 2000-01-01 |
| 4 | Mistral: Mistral Small 4 | 36.1 | model-only independent_benchmark | 2000-01-01 |
what this result means
BigCodeBench bigcodebench-instruct — practical Python coding benchmark.
This benchmark contributes direct public evidence. Read its scope before generalizing the result.
A win here is a win on BigCodeBench Instruct. Broad task pages require independent corroboration before naming a general winner.
source record
category: coding
metric: pass_at_1
matched models: 4
latest source date: 2000-01-01
direction: higher is better