EN
20 October 2026 · Jakub M.
How much memory does a language model need? A table from 7B to 405B
Will a 70B model fit in 64 GB? What about 235B? We worked out how much memory language models of different sizes take, how much the context adds, and which Mac is enough for what.
Whether a model fits in memory comes down to simple arithmetic. Multiply the number of parameters by the bits per parameter and divide by eight. A 70-billion-parameter model at full FP16 precision (16 bits) takes 140 GB, about half that at 8-bit and about a quarter at 4-bit. For local models, 4-bit quantisation is now the norm.
How much the weights take
| Size | Example models | 4-bit | 8-bit | FP16 (unquantised) |
|---|---|---|---|---|
| 7–8B | Llama 3.1 8B, Qwen3 8B | ≈ 4.5 GB | ≈ 8.5 GB | ≈ 16 GB |
| 14B | Qwen3 14B | ≈ 8 GB | ≈ 15 GB | ≈ 28 GB |
| 27–32B | Qwen3.5 27B, Qwen3 32B | ≈ 18 GB | ≈ 34 GB | ≈ 64 GB |
| 70B | Llama 3.3 70B | ≈ 39 GB | ≈ 74 GB | ≈ 140 GB |
| 117B | gpt-oss-120b | ≈ 66 GB | ≈ 124 GB | ≈ 234 GB |
| 235B | Qwen3 235B-A22B | ≈ 132 GB | ≈ 250 GB | ≈ 470 GB |
| 405B | Llama 3.1 405B | ≈ 228 GB | ≈ 430 GB | ≈ 810 GB |
Model weights only, rounded. Add memory for the context and 1–2 GB of overhead to each figure. In practice 4-bit quantisation takes about 4.5 bits per parameter, because the scales are stored too.
The context costs extra memory
Besides the weights, the model keeps the text it is working on in memory, the so-called KV cache. It grows with every token of context. For Llama 3.1 8B it is about 128 KB per token, so a 32,000-token context adds about 4 GB and the full 128,000 about 16 GB. For Llama 3.3 70B it is 320 KB per token: 32k means 10 GB extra, 128k a full 40 GB.
Newer models are more frugal. On our Mac Studio, gpt-oss-120b grew by only 1.5 GB between an empty context and 32k, because half of its layers only look at the last 128 tokens. Gemma 4 31B grew by 4.9 GB. The detailed numbers are in our measurement.
MoE models: memory for the whole, speed for a part
Mixture-of-experts (MoE) models such as gpt-oss-120b or Qwen3 235B-A22B use only part of their parameters for each token, which usually makes them much faster than a dense model of the same total size. But the whole model has to be in memory. A 235B MoE model with 22 billion active parameters therefore takes as much space as a 235B one, while its speed is closer to that of a much smaller model. How close depends on the architecture, expert routing, quantisation, context length and the software you run it in.
How much of the Mac’s memory the model really gets
A Mac has no separate graphics memory: the CPU and GPU share one pool. By default, though, macOS does not let the GPU take all of it and keeps a reserve for the system. You can raise the limit with a command that lasts until the next restart:
sudo sysctl iogpu.wired_limit_mb=240000The value is in megabytes. It is wise to leave the system at least 8–16 GB. On a rented Mac your account has no admin rights, so we set this limit for you on request.
| Mac memory | Roughly available to the model | What fits comfortably |
|---|---|---|
| 16 GB | 10–11 GB | 7–8B at 4-bit |
| 32 GB | 21–24 GB | 14B at 4-bit, 27–32B with a shorter context |
| 64 GB | 48 GB | 70B at 4-bit with a medium context, 32B at 8-bit |
| 128 GB | 96 GB | 117B MoE (gpt-oss-120b) at 4-bit, 70B at 8-bit |
| 256 GB | 192 GB, up to ~240 GB after raising the limit | 235B at 4-bit, 405B at 3-bit, or several models at once |
Estimate based on the default macOS limit for the GPU; the exact value varies with the system version.
How much quality quantisation costs
How much quality a quantised model loses depends on the model and the task, so test it on your own work. As a rough guide, 8-bit versions are usually very close to full precision, and 4-bit versions lose a little more and are the common compromise for running locally. Below 4 bits the drop tends to be more noticeable, especially with smaller models. With very large models such as 405B, a 3-bit version can still make sense, because a bigger model with coarser quantisation sometimes beats a smaller one at higher precision.
In short
- At 4-bit, count roughly 0.56 GB per billion parameters, plus the context and a few GB of overhead.
- A long context can take tens of GB, much less with newer models.
- With MoE models, the total size decides the memory. The active part has a big influence on speed, but it is not the only factor.
If your own Mac is not enough, our Mac Studio M5 Ultra with 256 GB should hold a 235B model at 4-bit, 405B at 3-bit, or several smaller models at once, as long as you leave room for the context and the system. These are estimates, so try your model before a longer rental. We can raise the GPU limit for you on request.