Local AI models by graphics card
On a PC, a model runs fastest when all of it sits in the graphics card's own memory. So the card's video memory decides what fits, whoever made the card.
On Windows, Task Manager shows it under Performance, GPU, as dedicated GPU memory. Two cards with the same memory fit the same models, so there is one page for each size.
On a Mac, memory is shared and the answer is different: models by Mac.
- 6 GB9 of 28 fit · RTX 2060, GTX 1660, RTX 3050 6GB and more
- 8 GB11 of 28 fit · RTX 4060, RTX 3060 Ti, RTX 5060 and more
- 10 GB12 of 28 fit · RTX 3080, Arc B570
- 11 GB11 of 28 fit · RTX 2080 Ti, GTX 1080 Ti
- 12 GB16 of 28 fit · RTX 4070, RTX 3060, RTX 5070 and more
- 16 GB17 of 28 fit · RTX 5070 Ti, RTX 4080, RTX 4060 Ti 16GB and more
- 20 GB17 of 28 fit · RX 7900 XT
- 24 GB17 of 28 fit · RTX 4090, RTX 3090, RX 7900 XTX and more
- 32 GB17 of 28 fit · RTX 5090, RTX 5000 Ada
- 48 GB18 of 28 fit · RTX A6000, RTX 6000 Ada