Apple's 4.3x is a prefill number, and one init-time check decides whether you get it

Apple announced the M5 Max and M5 Ultra Mac Studio on Monday: up to 512GB of unified memory, 1.2TB/s of memory bandwidth, Neural Accelerators in every GPU core, and the headline claim of "up to 4.3x faster AI performance." Pre-orders are open, machines ship September 22, 512GB configs in late October.

The 4.3x is real. It is also, specifically, a prompt-processing number — and whether your machine delivers it comes down to a capability check your inference runtime runs once at startup and reports in a log line almost nobody reads.

What 4.3x measures

Apple's own footnoted benchmarks are unambiguous about where the gain lives. For M5 Ultra: "up to 9.8x faster LLM prompt processing in LM Studio when compared to Mac Studio with M1 Ultra, and up to 4x faster than M3 Ultra." For M5 Max: 10.7x against M1 Max, 3.9x against M4 Max. Every LLM figure in the release is a prompt processing figure.

That is the honest place to put it. The M5 generation's addition is matmul hardware — Neural Accelerators in each GPU core, on an Ultra chip for the first time — and prefill is compute-bound, so dedicated matmul units move it a lot. Decode is not compute-bound. Token generation streams the active weights out of DRAM once per token, so the number governing your tokens/sec is 1.2TB/s of memory bandwidth, which Apple describes as "50 percent higher than before." Fifty percent, not 4.3x.

So the mental model for an M3 Ultra owner reading this announcement is roughly 4x on time-to-first-token and roughly 1.5x on generation. Which is a good trade, because prefill is where long-context RAG and agent loops actually spend their wall clock. It's just not the trade the headline implies.

The check that decides whether you get any of it

The accelerators are reached through Metal 4's tensor APIs. llama.cpp has upstream support (PR #16634) that probes for them at device init and enables the fast path on M5/A19-class hardware. When the probe passes, the log says has tensor = true. When it fails, you get this instead:

ggml_metal_device_init: - the tensor API is not supported
in this environment - disabling

and the model loads anyway, runs anyway, and generates at exactly the same speed as before. Nothing errors. Nothing warns.

There is a still-open LM Studio issue documenting this on an M5 Max MacBook Pro (128GB, macOS 26.5, runtime llama.cpp-mac-arm64-apple-metal-advsimd@2.21.0): the bundled runtime fails the probe on every GGUF load, while a stock upstream llama.cpp binary on the same machine and the same model files passes it. The reporter's diagnosis is that the shipped Metal library was built against a pre-Metal-4 SDK or deployment target, so the test kernels fail to compile at init — the code is there, the probe just never succeeds.

Their A/B, via llama-bench -fa 1 -p 2048 -n 128 -r 2 with the path toggled by GGML_METAL_TENSOR_DISABLE=1:

These are the reporter's numbers, not mine — I don't have M5 hardware to reproduce them on. But the shape is exactly what the architecture predicts, which is what makes them credible: the matmul units move prefill 2–2.4x and leave decode alone, because decode was never waiting on matmul.

Why the GGUF path fails and the MLX path doesn't

The asymmetry in that issue is the part worth stealing as a design lesson. On the same LM Studio install, the MLX side is fine: it ships an explicit M5-targeted nax runtime pack. The GGUF side ships one universal Metal build and asks it to detect its own capabilities at runtime.

Both are legitimate strategies. But runtime feature detection only works if the detection can distinguish "this hardware lacks the feature" from "this binary was built wrong" — and here it can't. The strings in the shipped dylib include tensor API disabled for pre-M5 and pre-A19 devices, so the runtime is doing a hardware-generation check and a compile-time self-test, and collapsing both outcomes into the same silent disable. A user on M5 hardware and a user on M2 hardware get identical logs.

Note the A19 half of that string, too. This isn't only a desktop problem. Every iPhone with an A19-class chip runs the same probe against whatever Metal library your app bundles.

What to actually do

Three things, in ascending order of effort.

Read the init log once. Load any model on M5-class hardware and grep for the tensor line. Thirty seconds, and it's binary — either has tensor = true or you're on the slow path.

Benchmark prefill, not chat feel. Chat feel is decode, and decode is unaffected, which is precisely why this hides:

$ llama-bench -m model.gguf -fa 1 -p 2048 -n 128 -r 2
$ GGML_METAL_TENSOR_DISABLE=1 llama-bench -m model.gguf -fa 1 -p 2048 -n 128 -r 2

If those two runs produce the same pp2048 number, the fast path was never on.

If you ship an app that bundles an inference runtime, this is your bug too. Verify your deployment target and SDK against Metal 4 on real M5/A19 hardware, and log the probe result somewhere you'll see it. Better: fail loud, or ship a hardware-targeted pack the way the MLX side does, instead of trusting one universal binary to be honest about itself.

Apple shipped genuinely impressive silicon this week, and 1.2TB/s against 512GB of unified memory is a real change in what runs locally. But a hardware multiplier is a ceiling, not a guarantee. The floor is whatever your software stack negotiates at startup — and right now, on the newest Apple hardware, that negotiation is one line in a log you've never read.

← Back to index