Skip to content
modelread

Loading devices…

Remembered on this device. Multi-GPU: search “2x”, “8x”, “node”.

Reading results

Each candidate (the best one and each of alternatives) has these fields:

FieldMeaning
verdictfits (in fast memory with headroom) · fits-tight (inside the budget's uncertainty) · offload (runs with part in system RAM: valid, slower) · no · unknown (not enough evidence; never treat as a fit)
repo, variant, formatThe file set chosen, e.g. unsloth/…-GGUF · Q6_K · gguf
decode_tpsSingle-user generation speed: lo, mid, hi, calibrated, basis. Quote the range. aggregate gives total tok/s at 1–32 concurrent sequences; tensor_parallel appears on multi-GPU nodes.
prefill_tpsPrompt processing speed, with at_8k_depth
max_ctx_fitLongest context that fits with this KV type
memory_bytesweights, KV cache, recurrent state, compute buffers
placement_bytesWhere it lives: vram, ram, disk_stream
budgetsUsable fast-tier and RAM budgets, with their uncertainty
bits_per_weightEffective quantization
confidence0–1; how much of the answer is verified vs estimated
launchCopy-paste llama.cpp / vLLM / MLX command for this device and context
reasons, unknowns, warnings, notesSay these to the user. “modified derivative” / “renamed derivative” means the repo changed the weights (fine-tune, merge, prune, abliteration).
evidence_tagsHow each part is known: memory VERIFIED (file headers), speed calibrated or not, vendor claims REPORTED
cost_per_million_output_tokensCloud presets with an hourly price

Speed badges

  • Measured: submitted runs exist for this device class and model family.
  • Calibrated estimate: the model class is fitted on measured devices.
  • Uncalibrated estimate: roofline outside measured classes; the range is wider.

How these are computed and how accurate they are: Methodology.

Askwhat should I run?
Browsebest models per device
Devicesall hardware
Connect your agentMCP · API
How the numbers workcalibration