ExecuTorch's Apple Silicon path is MLX and hand-written Metal. Not Core ML, not MPS.

Meta's ExecuTorch team published their Muse Glimmer on-device writeup on Aug 10, and ExecuTorch 1.4.1 hit PyPI on Aug 14. Read the two together and the Apple Silicon backend question gets a clear, slightly awkward answer.

Here is how the post describes lowering:

you export to ExecuTorch, and the framework handles backend-specific lowering, Triton on CUDA, MLX-native and custom Metal on Apple silicon.

And more specifically:

MLX — RMSNorm, RoPE, SDPA, KV-cache updates, and quantized linear operations are lowered to MLX-native or custom Metal implementations.

Core ML is not mentioned in the post. MPS is not mentioned in the post. The flagship Apple-silicon demo for PyTorch's own on-device runtime goes through a research framework Apple publishes on GitHub and a pile of hand-written Metal, and routes around both of the delegates Apple ships in the OS.

The build reflects it:

# NVIDIA CUDA (Linux or Windows)
(cd examples/models/muse-glimmer && cmake --workflow --preset muse-glimmer-cuda)
# Apple Silicon (macOS)
(cd examples/models/muse-glimmer && cmake --workflow --preset muse-glimmer-mlx)

Two presets. CUDA and MLX. That's the shipping matrix.

The K-quant repacking trick is the part worth stealing

The quantization plumbing is the most interesting part of the post, and it's useful regardless of what you think of ExecuTorch. They export from GGUF checkpoints directly and map K-quants onto backend-native formats:

We map Q4_K/Q5_K/Q6_K to packed INT4/5/6 with dp4a GEMV kernels on CUDA, and to repacked or fused Metal kernels on MLX.

On MLX, for performance, at repack time we merge adjacent sub-blocks whose scale and min are identical into a larger group size, up to 128, whenever the merge is lossless.

That second one is a nice piece of work. K-quants use small sub-blocks with per-sub-block scale and min; MLX's affine quantization is faster at larger group sizes because you touch less scale/bias metadata per unit of weight. The obvious options are both bad — requantize and take a quality hit, or eat the small-group cost and be slow. Instead they scan for adjacent sub-blocks that happen to share scale and min, and merge those for free. In practice K-quant checkpoints have a lot of them.

This is the kind of format-bridging that never shows up in a benchmark chart and largely determines whether your GGUF runs at a reasonable speed.

Reported result: on an M5 Pro with 64 GiB, Muse Glimmer text-image does 21.6 tok/s solo and 33.0 tok/s with DFlash speculative decoding — a 52.8% improvement without quality regression. For a 30B model at roughly 4-bit (under 20 GB for the LM), that's a usable interactive rate. Note it's the only hard number stated in the post's text; the A100-vs-M5-Max comparison in Figure 3 is chart-only, so don't quote it.

The awkward part

ExecuTorch's MPS backend was marked deprecated back in v1.2.0 with an explicit target: "The MPS backend is deprecated and will be removed in a future release (v1.4.0)." By v1.3.1 that had softened to "remains deprecated and is expected to be removed in a future release" — the version target quietly dropped. And the live PyPI description for 1.4.1, dated Aug 14, still says: "[macOS only] Core ML and MPS backend are also linked into the prebuilt module."

So MPS is not removed. It's deprecated, unreferenced in the flagship post, and still in the wheel.

Core ML is in a different position — actively maintained (v1.3.1 shipped a profiler crash fix, partitioner skips for unsupported random and argmin/argmax patterns, iOS 18 quantization error hints) but conspicuously absent from the LLM story.

Which tracks. Core ML's delegate is genuinely good at what it was designed for: fixed-shape vision and audio graphs you want resident on the ANE. It has never been the right vehicle for a KV-cached autoregressive decoder with dynamic sequence lengths, and the industry has now voted on that twice — once through llama.cpp choosing raw Metal, and now through ExecuTorch choosing MLX plus custom kernels.

One operational note if you go the ExecuTorch route: CUDA exports must be done on the same GPU architecture that will run the artifact — "For the best results, export on the same GPU architecture that will run the artifact." Plan your build machines accordingly.

← Back to index