EN
How much memory does an AI model need?
Pick how much memory your Mac has to see which open language models will run on it. Or pick a model to see how much memory it needs.
We assume 4-bit quantization, common for local models, and a context of up to about 32,000 tokens: parameters × 4.5 bits ÷ 8, plus 10 % and 2 GB for context and runtime. By default a Mac gives the GPU about 67 % of its memory (up to 36 GB) or 75 % (above 36 GB); the limit can be raised. These are estimates: actual use depends on the software and context length. On our Mac Studio with 256 GB and macOS 27.2.1 the GPU limit is 240 GB, so the calculator uses that for 256 GB.
Model data: Google Gemma, Wikipedia: Qwen, Nemotron, Kimi, OpenRouter, ComputingForGeeks.
How the calculator estimates memory
A language model needs memory for three things: its weights, the context it is working with, and the runtime around it. The calculator adds them up as follows.
Weights
With 4-bit quantization each parameter takes about 4.5 bits including the scaling factors, so roughly 0.56 GB per billion parameters. A 70B model needs about 39 GB just for its weights, a 120B model about 66 GB.
Context and KV cache
For every token in the context the model keeps keys and values in memory, the so-called KV cache. How much it takes depends on the model’s architecture and grows with the length of the context. The calculator adds 10 % and 2 GB to the weights, which covers a context of up to roughly 32,000 tokens for most models. For much longer contexts, plan for more.
Why a Mac cannot give all its memory to the model
On Apple Silicon the CPU and GPU share one unified memory, and macOS keeps part of it for the system. How much the GPU may use by default depends on the Mac and the macOS version. On our Mac Studio with 256 GB and macOS 27.2.1, Metal reports 240 GB, so everything except 16 GB, and a 224 GB model ran without changing any setting. Older macOS versions and smaller Macs typically allowed about two thirds to three quarters, and for those the calculator keeps this more cautious rule. The limit can be raised (the iogpu.wired_limit_mb setting) at the cost of memory for the system and other apps, which is why the calculator shows “With raised limit” separately.
4-bit, 8-bit or 16-bit
8-bit quantization needs about twice the memory of 4-bit, 16-bit (BF16) about four times. 4-bit is the usual compromise for running models locally: for most models the loss of quality is small, below 4 bits it grows faster. More in our article on how much memory a language model needs.
Estimate versus measurement
| Model | Calculator estimate | Measured peak | Smallest Mac where it fits |
|---|---|---|---|
| Qwen3 30B-A3B | 20.9 GB | 18.7 GB | 32 GB |
| Llama 3.3 70B | 45.7 GB | 43.3 GB | 64 GB |
| gpt-oss-120b | 74.4 GB | 64.5 GB | 128 GB |
Measured with mlx-lm 0.32.2 and an 8,192-token prompt on a Mac Studio M5 Ultra, 11 November 2026. gpt-oss-120b uses the MXFP4 format, which is a little smaller than the 4.5 bits the calculator assumes.
The estimates are deliberately on the safe side: in our measurement all three models needed a little less. If you only need a large Mac for a while, compare renting and buying a Mac Studio M5 Ultra. For Qwen3.5 397B the calculator uses our measured peak with a 32k context (227.5 GB) instead of the estimate, see our test of this model.