Benchmarks
The profile matrix
A profile is the unit of comparability. Two results share a rank only when the category, tier, lane, profile version, runtime, GPU count and parallelism all match. Everything else is shown side by side but never ranked together.
LLM
Ranked on decode throughput under TTFT and inter-token latency gates.BF16BF16. Runs on every supported GPU — the fair cross-generation baseline.
INT4AWQ INT4. Ampere and up. A larger model in the same VRAM.
NVFP4NVFP4. Blackwell only: RTX 5090, RTX PRO 6000, DGX Spark.
Entry12 GB+
4B · Qwen3-4B-Instruct
0 resultsAwaiting results
8B · Qwen3-8B-AWQ
0 resultsAwaiting results
8B · Qwen3-8B-NVFP4
0 resultsAwaiting results
Enthusiast24 GB+
8B · Qwen3-8B
0 resultsAwaiting results
32B · Qwen3-32B-AWQ
0 resultsAwaiting results
32B · Qwen3-32B-NVFP4
0 resultsAwaiting results
Frontier64 GB+
32B · Qwen3-32B
0 resultsAwaiting results
72B · Qwen2.5-72B-Instruct-AWQ
0 resultsAwaiting results
72B · Qwen2.5-72B-Instruct-NVFP4
0 resultsAwaiting results
Vision
Ranked on images per minute, behind a deterministic answer-accuracy gate.BF16BF16. Runs on every supported GPU — the fair cross-generation baseline.
INT4AWQ INT4. Ampere and up. A larger model in the same VRAM.
NVFP4NVFP4. Blackwell only: RTX 5090, RTX PRO 6000, DGX Spark.
Entry12 GB+
4B · Qwen3-VL-4B-Instruct
0 resultsAwaiting results
Not defined
Not defined
Enthusiast24 GB+
8B · Qwen3-VL-8B-Instruct
0 resultsAwaiting results
Not defined
Not defined
Frontier64 GB+
32B · Qwen3-VL-32B-Instruct
0 resultsAwaiting results
Not defined
Not defined
Diffusion
Ranked on seconds per denoising step. Images per minute is derived from it.BF16BF16. Runs on every supported GPU — the fair cross-generation baseline.
INT4AWQ INT4. Ampere and up. A larger model in the same VRAM.
NVFP4NVFP4. Blackwell only: RTX 5090, RTX PRO 6000, DGX Spark.
Entry12 GB+
4B · stable-diffusion-xl-base-1.0
0 resultsAwaiting results
Not defined
Not defined
Enthusiast24 GB+
8B · stable-diffusion-3.5-large
0 resultsAwaiting results
Not defined
Not defined
Frontier64 GB+
12B · FLUX.1-schnell
0 resultsAwaiting results
Not defined
Not defined
Why a leaderboard can stay hidden
A profile is runnable as soon as it exists, but its leaderboard only appears once two independent systems have verified results on it. A table with one row is not a comparison, and publishing it would imply a ranking that the data cannot support.
Profiles at version 0.x are release candidates: the model revision, runtime version and harness commit are still being pinned against the tier's minimum-VRAM reference system. Results collected against a 0.x profile are ranked only against that same version.