Measurements

What exists today

8 measured cells across 5 Mac configurations, last rolled up November 5, 2026. The catalog has 44 models and the site judges each on 59 Mac configurations. The estimate's constants were fitted on October 11, 2026; the catalog is from November 4, 2026.

Measured

Outmake times every answer a local model gives: tokens written a second and tokens read a second, as llama.cpp's server reports them, and every picture: seconds a step. With Send Measurements and Errors on (Settings › Privacy), those numbers go to Outmake's server with the chip, GPU core count, memory size, macOS version, the model's catalog name and size, the engine build, and the speed the app had predicted beside the one it measured. Nothing else: no prompt, no reply, no file path, no identifier, no timestamp finer than a day. The exact record is shown in the app before anything is sent.

Each night the server groups records by model, size, chip, memory and engine, drops the top and bottom tenth, and publishes the median. Every published number carries its sample count, from the first result. A number from fewer than three answers shows only in its table row, never in a page's title or summary. The whole table is one file: dataset.json, and one file per model under /v1/models/.

The sample is self-selected and unaudited: people who turned the switch on, running whatever they run, on whatever else was open. Records outside what a chip can do are refused at the door; a record that disagrees with itself (memory that cannot hold the model it names) is dropped. That is the whole defence, and it is written here so you can weigh it.

Estimated

Until a cell is measured, the site predicts it. Writing speed is the time one token takes: the active weights read at the chip's memory bandwidth, a fixed cost per layer, and attention over the chat so far, quoted 4,096 tokens in. Reading speed is the chip's GPU cores against the parameters each token wakes. The constants are fitted, by a script anyone can run, to 108 measured llama-bench rows from llama.cpp's own Apple silicon table (every M1 to M6 chip), four mixture-of-experts rows from discussion 15396, and depth sweeps from XiongjieDai and LocalScore.

How wrong: on the rows it was fitted to, the median error is -1% and the mean error 4.6%, the worst row 18.3%. On other people's 2026 runs it was not fitted to, M2, M4, M5 and M6 land within about 10% once the chat's length is accounted for. A mixture-of-experts model rests on four rows. Prompt speed on M5 and later assumes the engine's M5 matrix kernels. Any chip newer than M6 is read as the newest of its tier. A page marks these as extrapolated.

Fit is the app's verdict, not a rule of thumb: the memory each size needs at each context length was measured by llama.cpp's calculator when the catalog was built, and is judged against the Mac's memory and the share of it the graphics side may use. The catalog lists every size's memory.

Reproduce a row

The reference test is llama-bench's own: 512 tokens read, 128 written, five repetitions, at an empty chat and 4,096 tokens in, with llama-bench's defaults and no speculative drafting. Peak memory is the process's maximum resident size. On any Mac with llama.cpp:

/usr/bin/time -l llama-bench -m model.gguf -p 512 -n 128 -d 0,4096 -r 5 -o jsonl

Measure This Mac, which runs this same test from inside Outmake, is not in the released app yet. When it is, a row from it and a row from the command above will be the same test on the same code.

Rules

• Measured and estimated never share a figure. A number is one or the other, and says so.
• No composite score. One number per test, with its conditions: Mac, memory, model, size, engine, context.
• Sample count beside every number, from the first result. Fewer than three answers stay out of titles and summaries. Trimmed median, never a mean.
• The same rules for every model and every chip. No editorial on a vendor.
• When the engine or the test changes, the version changes with it, and old and new rows are not compared.
• A limitation found is written here, not removed from the data.

The data

Published under CC BY 4.0: use it, reprint it, build on it, with a link to outmake.app. The raw rows are never deleted. The fit constants, the fitting script and the measured rows it was fitted to are in the app's repository with the app.

To add your Mac: run a local model in Outmake with Send Measurements and Errors on. Your Mac's numbers correct the estimate for everyone with the same Mac, including the one Outmake shows you.