Check which models each candidate machine can run, before the purchase order
Teams that want to run models on their own hardware ask procurement for a GPU or a high-memory Mac. Vramwise turns the choice into a table: for each candidate machine and the models the team names, whether it fits, at which quantization, with how much context and roughly how fast.
How IT procurement teams use Vramwise
- Collect the Hugging Face model ids the team plans to run.
- Check each model on each candidate (for example a 24 GB GPU, a 48 GB GPU, a 64 GB or 128 GB Mac).
- Record the best quantization that fits at the context length the team needs.
- Share the Vramwise links with the request, or paste the Markdown tables (Pro) into the purchase justification.
Speeds are estimates from memory bandwidth, not benchmarks; ask vendors for measured numbers on the final shortlist.
Offload estimates (VRAM plus system RAM) and multi-GPU setups are included for PCs.
Qwen3-8B on an RTX 4090 24GB at 8K context: 18.6 GB of 24.0 GB usable at BF16
| Quantization | Weights | KV cache | Total | Fits? | Max context | Speed (tok/s) |
|---|---|---|---|---|---|---|
| BF16 / FP16 (16 bits) | 16.4 GB | 1.21 GB | 18.6 GB | Fits | 40K | 29–46 |
| Q8_0 / FP8 / MLX 8-bit (8.5 bits) | 8.70 GB | 1.21 GB | 10.9 GB | Fits | 40K | 51–81 |
| Q6_K (6.56 bits) | 6.72 GB | 1.21 GB | 8.92 GB | Fits | 40K | 64–102 |
| Q5_K_M (5.69 bits) | 5.83 GB | 1.21 GB | 8.03 GB | Fits | 40K | 72–115 |
| Q4_K_M / MLX 4-bit (4.85 bits) | 4.97 GB | 1.21 GB | 7.17 GB | Fits | 40K | 82–131 |
| IQ4_XS / AWQ / GPTQ 4-bit | 4.35 GB | 1.21 GB | 6.56 GB | Fits | 40K | 91–145 |
Source: Rows transcribed from the published Vramwise screenshot https://cybermaxtools.com/store/img/vramwise-1.webp (live app, Qwen3-8B on RTX 4090 24GB). Estimates, not measurements.
FAQ
Can it size servers for many users?
It estimates single-user speed. For multi-user serving, use it to check fit, then load-test.
Why not ask the vendor?
Do, for the final quote. Vramwise gives you an independent first answer in a minute, for free.
Vramwise: full guide with prices and alternatives · All use cases · Store