Home / Will it run? / Qwen2.5 Coder 7B Instruct
Alibaba Qwen · Qwen2.5 Coder
Will Qwen2.5 Coder 7B Instruct run on my GPU?
7.61B parameters, 32,768-token context, Apache-2.0. At Q4_K_M the weights come to about 4.3 GiB; add 1.5 GiB for the runtime and you need roughly 5.8 GiB of VRAM before context.
Weights by quant
GiBQ4_K_M · the everyday quant4.3 GiB
Q8_0 · near-lossless7.5 GiB
FP16 · full weights14.2 GiB
Runtime reserve1.5 GiB
Card by card
17 of 17 fit at Q4| Card | VRAM | Q4_K_M | Q8_0 | FP16 | Headroom at Q4 |
|---|---|---|---|---|---|
| GeForce RTX 5090NVIDIA | 32GB | Yes | Yes | Yes | 26.2 GiB |
| GeForce RTX 3090 (used)NVIDIA · used market | 24GB | Yes | Yes | Yes | 18.2 GiB |
| GeForce RTX 4090 (used)NVIDIA · used market | 24GB | Yes | Yes | Yes | 18.2 GiB |
| Radeon RX 7900 XTXAMD | 24GB | Yes | Yes | Yes | 18.2 GiB |
| GeForce RTX 5060 Ti 16GBNVIDIA | 16GB | Yes | Yes | Tight | 10.2 GiB |
| GeForce RTX 5070 TiNVIDIA | 16GB | Yes | Yes | Tight | 10.2 GiB |
| GeForce RTX 5080NVIDIA | 16GB | Yes | Yes | Tight | 10.2 GiB |
| Radeon RX 7800 XTAMD | 16GB | Yes | Yes | Tight | 10.2 GiB |
| Radeon RX 9060 XT 16GBAMD | 16GB | Yes | Yes | Tight | 10.2 GiB |
| Radeon RX 9070AMD | 16GB | Yes | Yes | Tight | 10.2 GiB |
| Radeon RX 9070 XTAMD | 16GB | Yes | Yes | Tight | 10.2 GiB |
| Arc B580Intel | 12GB | Yes | Yes | Offload | 6.2 GiB |
| GeForce RTX 3060 12GB (used)NVIDIA · used market | 12GB | Yes | Yes | Offload | 6.2 GiB |
| GeForce RTX 4070 SuperNVIDIA · used market | 12GB | Yes | Yes | Offload | 6.2 GiB |
| GeForce RTX 5070NVIDIA | 12GB | Yes | Yes | Offload | 6.2 GiB |
| Arc B570Intel | 10GB | Yes | Tight | Offload | 4.2 GiB |
| GeForce RTX 5060NVIDIA | 8GB | Yes | Offload | Offload | 2.2 GiB |
Headroom is what remains for the KV cache after weights and the 1.5 GiB reserve. How far it stretches depends on the architecture: grouped-query models are frugal, older dense models are not. If the table says tight, plan on a shorter context or a lower quant. Parameter count from the model's own card.