Benchmarks / LLM /Entry / BF16
LLM · Entry · BF16
A 4B-class model at BF16 on 12 GB. The universal baseline: every supported NVIDIA GPU from an RTX 3060 12 GB upward runs this identically, so it is the one leaderboard that spans the whole hardware range.
Best Decode
—tok/s
Ranked results
0
0 distinct systems
GPU models
0
Minimum VRAM
12 GB
~14 min run
Best result per GPU
Decode · tok/sNo results yet for this view.
Show the values behind this chart (0 rows)
| GPU | GPUs | Samples | Best (tok/s) | Mean (tok/s) |
|---|
Leaderboard
Including unranked results →No results match these filters yet.
What is pinned
Everything below is fixed by the profile and checked on every submission. A result whose runtime flags differ from these is published, but never ranked.
Model
- Repository
- Qwen/Qwen3-4B-Instruct
- Revision
- pending freeze
- Precision
- bf16
- Parameters
- 4 B
- Licence
- Apache-2.0
Runtime
- Engine
- vllm 0.26.0
- Harness
- aiperf
- dtype
- bfloat16
- max-model-len
- 16384
- gpu-memory-utilization
- 0.9
- max-num-seqs
- 16
- enforce-eager
- false
- disable-log-requests
- true
- tensor-parallel-size
- 1
- swap-space
- 0
Workloads
- interactive
- input_tokens=1024 output_tokens=256 concurrency=1 input_source=fixed_synthetic input_seed=20260801
- concurrency
- input_tokens=1024 output_tokens=256 concurrency_sweep=1/2/4/8 input_source=fixed_synthetic input_seed=20260801
- longcontext
- input_tokens=8192 output_tokens=1024 concurrency=1 input_source=fixed_synthetic input_seed=20260801
Ranking
- Ranked on
- Decode (tok/s, higher is better)
- Gate
- TTFT p95 ≤ 3000
- Gate
- Inter-token p95 ≤ 200
- Gate
- Error rate ≤ 0
- Also shown
- Prefill, TTFT p50, TTFT p95, Inter-token p95, Peak throughput @c8, Energy / token, Model load
Design notes
- Release candidate. The runtime and harness versions are pinned; the model revision is not frozen until the bake-off confirms fit on the tier's minimum-VRAM reference system, so this leaderboard is provisional.
- gpu-memory-utilization is 0.9 rather than the usual 0.95 because a 12 GB card frequently drives a display. If the bake-off shows 0.9 still fails with a desktop session attached, max-model-len drops before the utilisation does.
- Decode and prefill throughput are reported separately. Decode tracks memory bandwidth; prefill tracks compute. Reporting only TTFT would hide the difference between a DGX Spark and an RTX 4090.
- Only single-GPU results are accepted at Entry tier. A multi-GPU system may run this profile, but the runner pins it to one device.