The numbers
Measured on 21 September 2026 on a MacBook Pro with an M1 Pro and 16 GB of memory, running Qwen3 4B in llama.cpp.
| What it wrote | Off | On |
|---|---|---|
| An edit to a code file | 30.5 words a second (about 41 tokens) | 69.1 words a second (about 92 tokens) |
| A story | 29.8 words a second (about 40 tokens) | 30.7 words a second (about 41 tokens) |
From one Mac. The size of the model file wasn’t written down with the run. Tokens are worked out from words at three words to four tokens.
Why the edit doubled and the story didn’t
This kind of speculative decoding guesses from what is already in the conversation. When a model edits a file, most of what it writes back is the file it was just given, so the guesses are right for long runs and the model confirms many words in one pass. A story repeats nothing, so there is nothing to guess from and the speed stays the same.
So it helps most with code edits, rewrites, and anything that quotes the page back. It doesn’t help with fresh prose, and in this test it didn’t slow it down.
The words are the same
A guess is kept only where it is the word the model would have written. The output is the model’s own either way. A model with a picture attached drafted the same way in the same test.
No second model
The older way to do this loads a small draft model beside the big one, which takes memory a 16 GB Mac doesn’t have to spare. Guessing from the conversation needs no second model and almost no memory.
In Outmake
Nothing to turn on. Every chat model Outmake runs on a Mac has it on. Two models on Outmake’s list, Qwen3.8 27B among them, carry a predictor of their own in the file, and Outmake uses that first.
In llama.cpp yourself
The same setting is one flag on llama-server (checked against build b11718, the one Outmake 0.5 ships):
llama-server -m your-model.gguf --spec-type ngram-modOutmake also runs with flash attention and an 8-bit cache for the conversation, which halve the memory a long chat takes.