EN
12 November 2026 · Jakub M.
Qwen3.5 397B on a single Mac Studio M5 Ultra: 224 GB, 62 tokens per second
A model with 397 billion parameters takes about 224 GB in 4 bits. We ran it on one Mac Studio M5 Ultra with 256 GB: it generated around 62 tokens per second, handled a context of 128,000 tokens and needed no change to macOS settings.
Our largest test so far was Qwen3 235B at about 133 GB. This time we went to the edge of the Mac’s memory: Qwen3.5 397B-A17B, a mixture-of-experts model with 397 billion parameters, of which about 17 billion are active for each token. In 4-bit MLX format it takes about 224 GB.
How we measured
- Hardware: Mac Studio with Apple M5 Ultra (36-core CPU, 80-core GPU) and 256 GB of unified memory, macOS 27.2.1.
- Software: Python 3.12, MLX 0.32.4, mlx-lm 0.32.2, model
mlx-community/Qwen3.5-397B-A17B-4bit. - Test: the built-in
mlx_lm.benchmarktool, a prompt of random tokens with a fixed seed and a warm-up run before each measurement. Prompts of 512 and 8,192 tokens with 128 generated tokens (three and two runs), then longer prompts up to 131,072 tokens with 64 generated tokens (one run each).
Results
| Prompt (tokens) | Prompt processing (tok/s) | Generation (tok/s) | Peak memory |
|---|---|---|---|
| 512 | 159 | 61.6 | 224.3 GB |
| 8,192 | 1,265 | 59.9 | 226.8 GB |
| 16,384 | 1,432 | 58.3 | 227.0 GB |
| 32,768 | 1,500 | 55.1 | 227.5 GB |
| 65,536 | 1,345 | 50.3 | 228.5 GB |
| 131,072 | 1,080 | 44.1 | 230.6 GB |
Qwen3.5 397B-A17B, 4-bit MLX (mlx-community), mlx-lm 0.32.2, macOS 27.2.1, measured on 12 November 2026. 512 and 8,192 tokens: 128 generated tokens, average of 3 and 2 runs; longer prompts: 64 generated tokens, one run. The 512-token prompt figure varied between runs (141 to 195 tok/s).
No settings to change
We expected the model not to fit with the default settings. Older macOS versions let the GPU use only about three quarters of memory, which on a 256 GB Mac would be around 192 GB. On our Mac with macOS 27.2.1, Metal reports a recommended limit of 240 GB (257.7 billion bytes), so all memory except 16 GB. The model ran with the default settings at the same speed as with the GPU limit set by hand, swap stayed under 45 MB and macOS reported normal memory pressure throughout. We got the same value of 240 GB on our second 256 GB Mac. We have updated our RAM calculator accordingly.
For renters this matters: they do not have administrator rights, so they could not raise the limit themselves. On these Macs they do not need to.
Long context costs little memory
From a 512-token prompt to a 131,072-token prompt, peak memory grew by only about 6 GB, from 224.3 to 230.6 GB. Even with the longest context, more than 25 GB stayed free for the system. Speed does drop as the context grows: generation went from 62 to 44 tokens per second, which is still comfortable for reading.
Bigger, yet faster
In our October test, Qwen3 235B generated 46 tokens per second with an empty context and 33 with a 16,384-token prompt. Qwen3.5 397B is almost twice the size, yet generates about 62 and 58 tokens per second in comparable situations. The reason is the smaller number of active parameters per token: about 17 billion instead of 22. The October test used an older software version and a slightly different method, so take the comparison as a guide only.
Power
During the test the chip (CPU and GPU) drew at most about 104 W, measured with powermetrics in 2-second samples. We do not give an average, because the measurement also included loading the model and pauses.
Limits of the test
We measured one model on one Mac, with single requests. A prompt of random tokens gives pure speed figures; with real tasks the speed may differ. The test does not measure the quality of the answers, and the 4-bit version may answer slightly worse than the full model.
Try it yourself
If you want to run a model of this size without buying a Mac for over €12,000, you can rent a Mac Studio M5 Ultra with 256 GB from one day.