Most AI runs in the cloud. Your query leaves your device, hits a server farm somewhere, and comes back with a response. It works, but your data travels, your privacy depends on someone else’s policies, and you’re one internet outage away from a broken feature.
Apple just changed the math on that.
With macOS 27, every Mac that supports Apple Intelligence ships with Core Advanced — wait, scratch that. With macOS 27, every supported Mac ships with Core Advanced, a 20-billion-parameter language model that lives entirely on the device. A parameter is roughly a unit of learned knowledge inside the model. Most models this size need about 15 gigabytes of RAM to operate. Apple’s version takes around 2.
The technique behind this is called mixture of experts. Instead of loading all 20 billion parameters at once, the model activates only 1 to 4 billion of them per prompt, depending on what the task needs. The remaining parameters sit dormant in flash storage until called for. Apple took this approach further than most by selecting experts once per prompt rather than swapping them in and out token by token, which keeps storage reads manageable on a device running on battery.
Testing on an M3 Max MacBook Pro showed memory usage around 2 gigabytes. An open model of the same class, built the conventional way, consumed about 15. Energy draw came in at roughly a third of comparable open models running on the same machine. The model stayed loaded in the background while a full suite of apps ran alongside it.
There are real tradeoffs. Math performance landed last among the four models tested in the analysis. Apple’s own documentation lists basic math, code generation, and logical reasoning as capabilities to steer away from. Core Advanced was built for short, natural language tasks: summarizing content, extracting entities, refining and classifying text, powering voice and dictation. A purpose-built tool with a clear scope, not a general-purpose assistant competing with GPT or Claude.
Apple built the model in collaboration with Google, a fact visible from the outside through the shared tokenizer architecture. The core engineering choices, particularly around flash-based inference and per-prompt routing, trace back to Apple’s own research published in 2023.
For businesses, the implications are concrete. On-device AI means sensitive customer data never leaves the device for supported tasks. No per-query cloud cost. No network latency on each call. For companies navigating data residency requirements or privacy commitments to clients, this shifts the conversation from “can we use AI safely?” to “which tasks belong on-device?”
The architecture also signals where hardware AI is heading. If a 20-billion-parameter model runs in 2GB today, the devices shipping in three years will carry substantially more capability at a fraction of the current footprint.
Want to explore how on-device AI could benefit your business? Let’s talk.