15 October 2026 · Jakub Mokříš
Mac Studio M5 Ultra with 256 GB and local AI: we measured six models
How many tokens per second does a Mac Studio with the M5 Ultra and 256 GB of unified memory actually deliver? We ran six popular open models from 9 to 120 billion parameters, with an empty and a long context, and recorded memory use as well.
Benchmarks of Apple Silicon for local language models usually show one number for one model, without saying how it was measured. We wanted numbers that are complete and repeatable, so we measured the same way across all models and wrote down the method.
What we measured on, and how
- Hardware: Mac Studio with Apple M5 Ultra, 256 GB of unified memory and a 4 TB SSD. It is the same machine we rent out.
- Software: macOS 27.0.1, Python 3.12 and mlx-lm 0.32, built on Apple’s MLX framework.
- Models: MLX conversions from the mlx-community organisation on Hugging Face, in 4-bit quantization. The gpt-oss models use MXFP4 for the expert weights (“MXFP4-Q8” variants), Gemma 4 31B is the QAT 4-bit version.
- Test: 512 prompt tokens followed by 128 generated tokens, with 0, 8,192 and 32,768 tokens already in the context. Runs with an empty context were done twice and averaged, the longer contexts once. The prompt consists of random tokens; speed does not depend on what the text says.
- Memory: the peak memory reported by MLX during the test, including the model weights and the context cache.
Results
| Model | Type | Size | Gen., empty | Gen., 8k | Gen., 32k | Memory, empty | Memory, 32k |
|---|---|---|---|---|---|---|---|
| Qwen3.5 9B | dense | 6.0 GB | 167 | 161 | 141 | 5.5 GB | 7.4 GB |
| gpt-oss-20b | MoE | 12.1 GB | 184 | 172 | 145 | 11.7 GB | 12.8 GB |
| Qwen3.5 27B | dense | 16.1 GB | 55 | 53 | 48 | 15.3 GB | 18.8 GB |
| Gemma 4 31B | dense | 28.8 GB | 31 | 27 | 21 | 26.6 GB | 31.5 GB |
| Qwen3.5 35B-A3B | MoE | 20.4 GB | 162 | 154 | 141 | 18.8 GB | 20.6 GB |
| gpt-oss-120b | MoE | 63.4 GB | 120 | 113 | 96 | 59.5 GB | 61.0 GB |
Generation speed in tokens per second. “Empty”, “8k” and “32k” is the number of tokens already in the context. Memory is the peak memory reported by MLX.
What the numbers say
Mixture-of-experts models are much faster
The fastest model in the test is not the smallest one. gpt-oss-20b generates 184 tokens per second and even gpt-oss-120b reaches 120. Both are mixture-of-experts (MoE) models: for each token they only use a small part of their parameters, about 3.6 billion for gpt-oss-20b and 5.1 billion for gpt-oss-120b according to OpenAI. Qwen3.5 35B-A3B works the same way with roughly 3 billion active parameters and reaches 162 tokens per second.
Dense models use all their parameters for every token, so they are slower: Qwen3.5 27B generates 55 tokens per second and Gemma 4 31B 31. For comparison, around 20 tokens per second is already faster than most people read, so even the slowest model in the test is comfortable for chat.
A long context costs 12 to 32 percent
With 32,768 tokens already in the context, generation slowed down by 12 to 21 percent for most models. Gemma 4 31B lost the most, from 31 to 21 tokens per second, about a third. If you work with long documents or large codebases, this is the column to look at.
256 GB leaves a lot of room
The largest model we tested, gpt-oss-120b, needed 61 GB including a 32k context, less than a quarter of the memory. The remaining space is enough for much larger models, for example Qwen3.5 122B-A10B (about 70 GB in 4-bit) or even Qwen3.5 397B-A17B, whose 4-bit files take 224 GB. It also allows running several models side by side, for example a large model for answers and a small one for fast routing or embeddings.
What we did not measure
- Prompt processing speed. Our short test of 512 tokens gave fluctuating results, roughly between 500 and 5,000 tokens per second depending on the model. That is enough to say reading a prompt is many times faster than generating, but not enough for exact numbers. We will add a longer measurement.
- Other runtimes. We only used MLX. llama.cpp, Ollama or GGUF models in LM Studio can give different results.
- Answer quality, power draw and the largest models (122B and 397B). These are next on our list.
Try it on your own workload
Numbers from someone else’s test never fully answer whether a machine is right for your work. That is why we rent out this exact Mac Studio, from one day, over SSH or remote desktop. If you are considering buying one, you can run your own models and data on it first.