Reading results
Each candidate (the best one and each of alternatives) has these fields:
| Field | Meaning |
|---|---|
verdict | fits (in fast memory with headroom) · fits-tight (inside the budget's uncertainty) · offload (runs with part in system RAM: valid, slower) · no · unknown (not enough evidence; never treat as a fit) |
repo, variant, format | The file set chosen, e.g. unsloth/…-GGUF · Q6_K · gguf |
decode_tps | Single-user generation speed: lo, mid, hi, calibrated, basis. Quote the range. aggregate gives total tok/s at 1–32 concurrent sequences; tensor_parallel appears on multi-GPU nodes. |
prefill_tps | Prompt processing speed, with at_8k_depth |
max_ctx_fit | Longest context that fits with this KV type |
memory_bytes | weights, KV cache, recurrent state, compute buffers |
placement_bytes | Where it lives: vram, ram, disk_stream |
budgets | Usable fast-tier and RAM budgets, with their uncertainty |
bits_per_weight | Effective quantization |
confidence | 0–1; how much of the answer is verified vs estimated |
launch | Copy-paste llama.cpp / vLLM / MLX command for this device and context |
reasons, unknowns, warnings, notes | Say these to the user. “modified derivative” / “renamed derivative” means the repo changed the weights (fine-tune, merge, prune, abliteration). |
evidence_tags | How each part is known: memory VERIFIED (file headers), speed calibrated or not, vendor claims REPORTED |
cost_per_million_output_tokens | Cloud presets with an hourly price |
Speed badges
- Measured: submitted runs exist for this device class and model family.
- Calibrated estimate: the model class is fitted on measured devices.
- Uncalibrated estimate: roofline outside measured classes; the range is wider.
How these are computed and how accurate they are: Methodology.