Speculative decoding is not a free 2x on Apple Silicon, and MoE makes it worse
Liquid AI released DSpark draft models for the LFM2.5 family on Wednesday. Three ~300M-parameter sidecar drafters (5 attention-only layers, a Markov-chain sequential head, a confidence head, block size 9) for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B, with day-one GGUF and a llama.cpp integration.
The dense results are good. On an M4 Max MacBook Pro, llama.cpp + Metal, FP16 GGUF, batch size 1, temp 0:
| Model | Mean speedup, M4 Max | Mean speedup, H100 |
|---|---|---|
| LFM2.5-1.2B-Instruct | 2.54x (138 → 350 tok/s) | 2.10x (656 → 1384 tok/s) |
| LFM2.5-2.6B | 2.27x (61 → 139 tok/s) | 2.67x (323 → 864 tok/s) |
| LFM2.5-8B-A1B (MoE) | 1.18x (90 → 106 tok/s) | 2.54x (418 → 1074 tok/s) |
Look at the last row, then look at its acceptance rate: 6.95 out of 10 on average, and 8.52 on MT-Bench — the highest acceptance numbers in the entire post. MT-Bench specifically went from 87 to 90 tok/s. A 1.04x speedup on an 85% acceptance rate.
This is the single most useful number Liquid published, and it's the one nobody will quote. It severs the link between acceptance rate and wall-clock speedup that most speculative-decoding writeups quietly assume. Their own explanation:
[…] the current MoE implementation in llama.cpp's Metal backend, and […] the fact that verifying k tokens activates more experts and thus more weight traffic than a single decode step.
Why the second clause is the important one
That second clause is the structural part, and it isn't a llama.cpp bug you can wait out.
Speculative decoding works because decode is memory-bound: you amortise one pass of weight streaming across k tokens. For a dense model, verifying 9 tokens streams the same weights as verifying 1. For an 8B-A1B MoE, each of the 9 candidate tokens can route to a different expert set, so the union of activated experts over the block is much larger than the single-token set. You pay for the amortisation up front in extra weight traffic. On an H100 with roughly 3.35 TB/s of HBM you don't notice. On an M4 Max at up to 546 GB/s you notice immediately.
The rule this implies: the sparser the model, the less speculative decoding buys you on unified memory — and the direction of the error is the opposite of what your intuition says. A high acceptance rate on an MoE is a signal that you're about to activate a lot of experts, not that you're about to go fast.
Two caveats before you plan a release around these numbers
First, the on-device measurements use FP16 GGUF weights. Nobody ships FP16 on device. At Q4, the baseline decode gets roughly 4x cheaper in weight traffic while the verification compute stays where it is, so the arithmetic intensity of the whole setup shifts toward compute-bound — exactly the regime where speculation stops paying. I'd expect the Q4 speedups to be materially lower than these, and I'd want to see that table before committing a drafter to a release build.
Liquid's own interactivity charts make the same argument in a different variable: on an H100, 2.6B performance "converges around bs=128 for block size 9," because concurrency and speculation are buying you the same thing. As arithmetic intensity rises you leave the memory-bound regime where speculation pays.
Second, the numbers were produced with experimental Metal kernels from an unmerged PR, not stock llama.cpp. Worth checking what you actually get on the build you're shipping.
What's actually good here
None of this makes DSpark a bad release. The dense on-device numbers are real, the quality guarantee is exact under greedy decoding by construction, and the 57% average function-call latency reduction for the 2.6B is the most convincing on-device agentic number I've seen this quarter — reasoning-before-tool-call is precisely where a user sits and waits.
But I want to name what's good here that has nothing to do with the models: they had a 1.04x result on their most-hyped architecture and shipped the table. Most vendors would have dropped the MoE row, kept "up to 3.18x" in the headline, and been technically honest. Publishing the number where the technique doesn't work is what makes the numbers where it does work believable.
If you're evaluating speculative decoding for an on-device deployment: test on your quantization, on your model's sparsity pattern, on your target silicon. Acceptance rate is not the metric. Tokens per second is.