[ Video ] Testing Apple Intelligence on my 16GB MacBook Pro
This is the 8th episode in the series where I’m testing a local AI model that runs on a 16GB MacBook Pro. Today I’m doing something special: I picked the model that Apple ships with Apple Intelligence (often called the Apple Foundation Model or AFM). It is already on your machine (Mac, iPhone, iPad) once you turn Apple Intelligence on. Apple does not distribute it the usual way (no Hugging Face weights, traditionally opaque), and there was no straightforward chat interface for it until recently.
I used an open-source tool apfel to get a normal chat interface:
Arthur-Ficial/apfel
It installs easily (Homebrew) and it provides a CLI and an OpenAI-compatible local server. No extra model download is needed.
I Ran The Two Standard Tests
First, the short question “What is a dolphin?” The answer appeared almost immediately. Across all the models I’ve run on this 16GB MacBook Pro, this was the fastest generation.
Second, the coding task: generate a small utility that counts words in files inside a folder. It defaulted to Python and produced a working script very quickly.
Because this is a rather unusual mode, a few technical notes. According to what I could find online, it is roughly a 3 billion parameter model. The space it occupies on disk is about 8 GB (Apple is opaque about the exact breakdown). Also, there is one caveat: some requests can be routed to Private Cloud Compute, so it is not always possible to know with high accuracy which ones stay fully on-device. On newer hardware (M3 and later) Apple can use a larger on-device variant that takes more space (up to around 12 GB).
Even the smaller model was impressive for speed and basic usability on a 16GB machine.
If you have other models that fit in 16GB and want them tested the same way, leave a comment. I read them.
And if you like this kind of content, like, share and subscribe to my YouTube channel!
Mentions
[ Video ] Testing Apple Intelligence on my 16GB MacBook Pro This is the 8th episode in the series where I’m testing a local AI model that runs on a 16GB MacBook Pro. Today I’m doing something special: I picked the model that Apple ships with Apple Intelligence (often called the Apple Foundation Model or AFM). It is already on your machine (Mac, iPhone, iPad) once you turn Apple Intelligence on. Apple does not distribute it the usual way (no Hugging Face weights, traditionally opaque), and there was no straightforward chat interface for it until recently. I used an open-source tool apfel to get a normal chat interface: Arthur-Ficial/apfel It installs easily (Homebrew) and it provides a CLI and an OpenAI-compatible local server. No extra model download is needed. I Ran The Two Standard Tests First, the short question “What is a dolphin?” The answer appeared almost immediately. Across all the models I’ve run on this 16GB MacBook Pro, this was the fastest generation. Second, the coding task: generate a small utility that counts words in files inside a folder. It defaulted to Python and produced a working script very quickly. Because this is a rather unusual mode, a few technical notes. According to what I could find online, it is roughly a 3 billion parameter model. The space it occupies on disk is about 8 GB (Apple is opaque about the exact breakdown). Also, there is one caveat: some requests can be routed to Private Cloud Compute, so it is not always possible to know with high accuracy which ones stay fully on-device. On newer hardware (M3 and later) Apple can use a larger on-device variant that takes more space (up to around 12 GB). Even the smaller model was impressive for speed and basic usability on a 16GB machine. If you have other models that fit in 16GB and want them tested the same way, leave a comment. I read them. And if you like this kind of content, like, share and subscribe to my YouTube channel!
I asked Claude to build something nice on Apple Intelligence (i.e. all local inference) and it made a nice mystery app in Swift, a game to solve a crime! No API keys, all local; built on top of Arthur-Ficial/apfel See the Pawprint repo: y-a-v-a/pawprints t.co
here's the repository if anybody wants to dig deeper - Arthur-Ficial/apfel