Outmake Models
Models

Meta Llama 3.1 8B

camenduru · July 2024

Get Hugging Face

A 8B chat model from camenduru.

Downloads
475K
Likes
757
Parameters
8B
Context
128K

Runs great on M4 · 16 GB

The Q4_K_S size from MaziyarPanahi is the one to get: a 4.7 GB download that uses 7.3 GB of memory, and writes about 7.6 words a second.

Meta Llama 3.1 8B runs well from a Mac with an M1 and 8 GB of memory. 59 of 59 Mac configurations run it well.

Compare with

The nearest models in size.

Sizes

From ThomasBaruzier/Meta-Llama-3.1-8B-Instruct-GGUF on Hugging Face. A smaller size is a smaller download and a little less accurate.

SizeDownloadMemory at 8KGet
IQ1_S2.0 GB2.7 GBestimatedOpen
IQ1_M2.2 GB2.9 GBestimatedOpen
IQ2_XXS2.4 GB3.1 GBestimatedOpen
IQ2_XS2.6 GB3.3 GBestimatedOpen
IQ2_S2.8 GB3.5 GBestimatedOpen
IQ2_M2.9 GB3.7 GBestimatedOpen
Q2_K_S3.0 GB3.7 GBestimatedOpen
Q2_K3.2 GB3.9 GBestimatedOpen
IQ3_XXS3.3 GB4.0 GBestimatedOpen
IQ3_XS3.5 GB4.2 GBestimatedOpen
Q3_K_S3.7 GB4.4 GBestimatedOpen
IQ3_S3.7 GB4.4 GBestimatedOpen
IQ3_M3.8 GB4.5 GBestimatedOpen
Q3_K_M4.0 GB4.7 GBestimatedOpen
Q3_K_L4.3 GB5.1 GBestimatedOpen
IQ4_XS4.4 GB5.2 GBestimatedOpen
Q4_0_4_44.7 GB5.4 GBestimatedOpen
Q4_0_4_84.7 GB5.4 GBestimatedOpen
Q4_0_8_84.7 GB5.4 GBestimatedOpen
Q4_04.7 GB5.4 GBestimatedOpen
IQ4_NL4.7 GB5.4 GBestimatedOpen

Run It Anywhere

Meta Llama 3.1 8B is an open model, so any app that runs GGUF files can run it. These take the Q4_K_S size from MaziyarPanahi straight from Hugging Face.

  • LM StudioMac, Windows, LinuxOpen
  • JanMac, Windows, LinuxOpen
  • MstyMac, Windows, LinuxOpen
  • OllamaMac, Windows, Linuxollama run hf.co/MaziyarPanahi/Meta-Llama-3.1-8B-Instruct-GGUF:Q4_K_S
  • llama.cppMac, Windows, Linuxllama-server -hf MaziyarPanahi/Meta-Llama-3.1-8B-Instruct-GGUF:Q4_K_S
  • Docker Model RunnerMac, Windows, Linuxdocker model run hf.co/MaziyarPanahi/Meta-Llama-3.1-8B-Instruct-GGUF:Q4_K_S
On a PC Graphics Card
Video memoryLargest size that fitsDownloadMemory at 8K
6 GBIQ4_NL4.7 GB5.4 GB
8 GBIQ4_NL4.7 GB5.4 GB
10 GBIQ4_NL4.7 GB5.4 GB
11 GBIQ4_NL4.7 GB5.4 GB
12 GBIQ4_NL4.7 GB5.4 GB
16 GBIQ4_NL4.7 GB5.4 GB
20 GBIQ4_NL4.7 GB5.4 GB
24 GBIQ4_NL4.7 GB5.4 GB
32 GBIQ4_NL4.7 GB5.4 GB
48 GBIQ4_NL4.7 GB5.4 GB

The largest size, of those that keep three bits a weight or more, whose memory at an 8K chat is under 92% of the card's; the rest is the desktop's. Estimated from the files, the same way as for a Mac; speed on a PC is not estimated yet. A size that doesn't fit the card still runs, slower, with part of it in ordinary memory.

On every Mac
MacSizeFitMemoryWrites
M6 · 16 GBQ4_K_SRuns great7.3 GB11
M6 · 24 GBQ4_K_SRuns great9.8 GB11
M6 · 32 GBQ4_K_SRuns great15 GB11
M5 · 16 GBQ4_K_SRuns great7.3 GB9.8
M5 · 24 GBQ4_K_SRuns great9.8 GB9.8
M5 · 32 GBQ4_K_SRuns great15 GB9.8
M5 Pro · 24 GBQ4_K_SRuns great9.8 GB19
M5 Pro · 48 GBQ4_K_SRuns great15 GB19
M5 Pro · 64 GBQ4_K_SRuns great15 GB19
M5 Max · 36 GBQ4_K_SRuns great15 GB28
M5 Max · 48 GBQ4_K_SRuns great15 GB37
M5 Max · 64 GBQ4_K_SRuns great15 GB37
M5 Max · 128 GBQ4_K_SRuns great15 GB37
M5 Ultra · 96 GBQ4_K_SRuns great15 GB69
M5 Ultra · 256 GBQ4_K_SRuns great15 GB69
M5 Ultra · 512 GBQ4_K_SRuns great15 GB69
M4 · 16 GBQ4_K_SRuns great7.3 GB7.6
M4 · 24 GBQ4_K_SRuns great9.8 GB7.6
M4 · 32 GBQ4_K_SRuns great15 GB7.6
M4 Pro · 24 GBQ4_K_SRuns great9.8 GB16
M4 Pro · 48 GBQ4_K_SRuns great15 GB16
M4 Pro · 64 GBQ4_K_SRuns great15 GB16
M4 Max · 36 GBQ4_K_SRuns great15 GB23
M4 Max · 48 GBQ4_K_SRuns great15 GB29
M4 Max · 64 GBQ4_K_SRuns great15 GB29
M4 Max · 96 GBQ4_K_SRuns great15 GB23
M4 Max · 128 GBQ4_K_SRuns great15 GB29
M3 · 8 GBIQ4_XSRuns well5.2 GB6.6
M3 · 16 GBQ4_K_SRuns great7.3 GB6.4
M3 · 24 GBQ4_K_SRuns great9.8 GB6.4
M3 Pro · 18 GBQ4_K_SRuns great7.3 GB9.4
M3 Pro · 36 GBQ4_K_SRuns great15 GB9.4
M3 Max · 36 GBQ4_K_SRuns great15 GB18
M3 Max · 48 GBQ4_K_SRuns great15 GB23
M3 Max · 64 GBQ4_K_SRuns great15 GB23
M3 Max · 96 GBQ4_K_SRuns great15 GB18
M3 Max · 128 GBQ4_K_SRuns great15 GB23
M3 Ultra · 96 GBQ4_K_SRuns great15 GB40
M3 Ultra · 256 GBQ4_K_SRuns great15 GB40
M3 Ultra · 512 GBQ4_K_SRuns great15 GB40
M2 · 8 GBIQ4_XSRuns well5.2 GB6.6
M2 · 16 GBQ4_K_SRuns great7.3 GB6.4
M2 · 24 GBQ4_K_SRuns great9.8 GB6.4
M2 Pro · 16 GBQ4_K_SRuns great7.3 GB12
M2 Pro · 32 GBQ4_K_SRuns great15 GB12
M2 Max · 32 GBQ4_K_SRuns great15 GB23
M2 Max · 64 GBQ4_K_SRuns great15 GB23
M2 Max · 96 GBQ4_K_SRuns great15 GB23
M2 Ultra · 64 GBQ4_K_SRuns great15 GB40
M2 Ultra · 128 GBQ4_K_SRuns great15 GB40
M2 Ultra · 192 GBQ4_K_SRuns great15 GB40
M1 · 8 GBIQ4_XSRuns well5.2 GB4.4
M1 · 16 GBQ4_K_SRuns great7.3 GB4.3
M1 Pro · 16 GBQ4_K_SRuns great7.3 GB12
M1 Pro · 32 GBQ4_K_SRuns great15 GB12
M1 Max · 32 GBQ4_K_SRuns great15 GB21
M1 Max · 64 GBQ4_K_SRuns great15 GB21
M1 Ultra · 64 GBQ4_K_SRuns great15 GB34
M1 Ultra · 128 GBQ4_K_SRuns great15 GB34

Writes is words a second, estimated from the Mac's memory speed and the model's size; how, and how wrong.

Packaged by

6 repositories on Hugging Face carry this model for llama.cpp. Outmake reads them all as one model.

Information

Maker
camenduru
Released
July 2024
Parameters
8B
Longest chat
128K tokens
Reads pictures
No
Licence
Not stated
Downloads this month
474,761
Likes
757
Sizes
21, 2.0 to 4.7 GB
Kind
Maker's release

Badge for the model card

Shows on Meta Llama 3.1 8B's Hugging Face card which Macs run it well, linked to this page. It updates every morning.

Runs on Mac badge

Files, downloads and likes read from Hugging Face on November 5, 2026 and refreshed every morning. Memory and speed are estimates until someone running Outmake measures this model on that Mac.

Your Computer

Every model is judged for this Mac. Both are under About This Mac, in the Apple menu.

Every Mac · Every graphics card

Civitai

Open in OutmakeHugging Face

New to Outmake? Download the free beta