How the numbers work
modelread answers from the repository itself, not from a parameter count. It lists the actual files, reads safetensors and GGUF headers over HTTP range requests (nothing is downloaded), and builds the architecture from the config: attention type per layer, KV heads, sliding windows, MoE routing, recurrent state, per-layer embeddings, vision towers.
Memory
Weights come from real file sizes. KV cache is computed per layer kind (full, sliding-window, linear/recurrent, MLA, shared-KV), so hybrid and SWA models are not overcounted. A device's usable budget subtracts the runtime context, the compute buffer and OS headroom, with an uncertainty band. When the fast tier is not enough, the placement moves MoE experts, then layers, to system RAM, and says so.
Speed
Decode is a roofline over the bytes each token actually streams (active experts only for MoE, gathered embeddings excluded) at the device's bandwidth and a calibrated efficiency, plus a fixed per-token overhead. Prefill is compute-bound, with attention cost growing with depth. Multi-user throughput uses a batching model, and multi-GPU nodes get a tensor-parallel estimate fitted to published serving tables. Every number is shown as a range, with a badge for how it is known.
Calibration
Each prediction is scored against a measurement before that measurement is used for fitting.
| Device | Source | What | Error |
|---|---|---|---|
| RTX 3090 | measured | decode: dense, hybrid and MoE | ±1–15% (learned per-architecture factors) |
| RTX 3090 | measured | prefill at 8K/16K depth | ≤2%; gemma-4 (head_dim 512) −6..+14% |
| RTX 4060 Laptop | measured | decode / max context / MoE offload placement | +1% / −0.4% / correct --n-cpu-moe |
| Jetson Orin Nano, Thor | measured | max context / decode | −3% / −3..−19% (conservative) |
| Ryzen 5900X, CPU only | measured | decode | +1..+10% |
| RTX 3090 concurrency sweep | measured | aggregate tok/s, 1–32 sequences | −1..+11% |
| RTX 3090 quant sweep | measured | IQ2/IQ3 kernel cost vs K-quants | learned ×1.24 |
| Phones | published | decode per SoC (arXiv 2607.05475, apple-silicon-llm-bench, LiteRT cards) | Gemma-4-E2B on iPhone 17 Pro −14% |
| 8× H100 / H200 / A100 | published | tensor-parallel per-user decode, TP2–TP8 (NVIDIA NIM / TensorRT-LLM) | −12..+15% |
Verdicts
- Fits
- Runs entirely in fast memory with headroom.
- Tight fit
- Fits, but within the uncertainty of the memory budget.
- Offload
- Runs with part of the model in system RAM: works, slower.
- Vendor-reported
- Only the vendor's own claim is available.
- Won't fit
- Does not fit as configured, even with offload.
- Unknown
- Not enough evidence to decide.
Speed badges
- Measured: submitted runs on that device class with this model family.
- Calibrated estimate: the model class has been fitted on measured devices.
- Uncalibrated estimate: a roofline outside measured classes, so the range is wider.
Quality
Quant quality is an expected loss from published KL-divergence tables, scaled by model size (bigger models lose less) and MoE active size. Benchmark percentiles compare a model to every indexed model with that benchmark; vendor-reported and independent scores are kept apart. Derivatives that change the weights (abliterated, merged, pruned, distilled, renamed fine-tunes) rank below faithful quants, with the reason shown.
Known limits
- MoE offload speeds carry ±50%.
- Uploader recipes vary ±50% in quality at the same quant label.
- Not modeled: NPU runtimes beyond listing them, speculative-decoding gains, phone thermal throttling, per-GPU fragmentation.
Make it better
Run modelfit bench --submit (llama-bench) or modelfit harvest (llama-server logs). Measurements change predictions for that device and teach the cross-device architecture and quant-kernel factors.