MiniCPM5-2B's 131k context is 5.25 GiB of KV cache on a phone. The config.json tells you more than the leaderboard.

OpenBMB shipped MiniCPM5-2B on September 7: 2,516,756,480 parameters, dense, plain LlamaForCausalLM, Apache 2.0, with GGUF, a 4-bit MLX build and a LiteRT-LM build in the same collection. The model card leads with an average of 53.9 over 34 benchmarks against 51.1 for Qwen3.5-4B. Skip that for a minute and open config.json, because that's where the shipping decisions are.

"hidden_size": 2048,
"intermediate_size": 6144,
"num_hidden_layers": 42,
"num_attention_heads": 16,
"num_key_value_heads": 2,
"head_dim": 128,
"max_position_embeddings": 131072,
"vocab_size": 130560,
"tie_word_embeddings": false

The KV cache arithmetic

Two KV heads at head_dim 128 is aggressive GQA (8:1), and it's the reason a 131k window is even printable on a model card. Per token, per layer, K and V in f16 cost 2 × 2 × 128 × 2 bytes = 1,024 bytes. Over 42 layers that's 43,008 bytes per token. So:

The Q4_K_M GGUF is 1.56 GB, the way Hugging Face counts it, so mind the units in the comparisons below: 5.25 GiB of cache is 5.64 GB decimal. At 16k context the cache is already 45% of the weights; at the advertised 131k it's 3.6x the weights, before activations, before your app, before the OS. -ctk q8_0 -ctv q8_0 halves it and nothing in this architecture makes that a bad trade. The honest phone number for this model is 8k to 16k, and you should size the context from this table rather than from the "131,072" on the card.

The embedding tax

21% of the parameters are embeddings: 130,560 × 2,048 × 2 (untied) = 534.8M, which is exactly the gap between the total and the 1,981,982,720 "non-embedding" figure OpenBMB quotes. The input side is free at decode time (one row lookup). The output head is not: every decoded token reads all 267M lm_head parameters. In a memory-bandwidth-bound decode loop, bytes touched per token is roughly non-embedding weights plus lm_head, about 2.25B parameters at whatever bit-width your quant gives them. A 130k vocab is nice for Chinese and code tokenization; it's also a fixed ~12% surcharge on every token you generate. Plan for it.

The other shape note: 42 layers at hidden 2048 is deep and narrow for 2.5B. Depth is sequential; on Metal every layer is another set of command-encoder dispatches on the decode critical path, and at batch size 1 that overhead doesn't amortize. Whether that matters versus a shallower 2B depends on your runtime's per-op cost, so bench llama-bench -n 128 on your target device, not a Mac.

Which "Intelligence Index" number

OpenBMB's announcement says MiniCPM5-2B scores 23 on the Artificial Analysis Intelligence Index. Artificial Analysis's own writeup, published the same day, says 15. Both are true: 23 was v4.1.1, 15 is v4.2, and AA states in the article that "scores across the two versions are not directly comparable." When you see an index score in a launch post this month, check the version. AA bumped the index twice in a week (v4.2 on September 4, v4.3 on September 7).

The parts of AA's evaluation that matter for a phone are these. It used 19k output tokens per index task, 11k of them reasoning, joint-lowest among the models AA compared and about a third of Ling 3.0 Tiny's 56k. At phone decode speeds that gap is the difference between an answer and a spinner. Its AA-Omniscience result is a non-hallucination rate of 78% earned by attempting only 29% of questions, with 8% accuracy; that's a model that declines rather than guesses, which is the right default for an offline assistant and a wrong one for a knowledge lookup. And it's a hybrid-thinking model: enable_thinking in the chat template switches modes. Ship it with thinking off by default and turn it on per request. AA's August mobile study showed that at a 60-second answer budget on an iPhone 17 Pro, reasoning-heavy 3B models fell to the bottom of the table because their reasoning never finished.

What to actually do

Pull the Q4_K_M GGUF and the litert-community/MiniCPM5-2B build, run both at 8k context on the oldest device you support, and record peak memory and tokens per second with thinking off. If tool calling is your use case, this is the model to test: 97.1 on τ²-Bench Telecom and 66.6 on BFCL v4 in OpenBMB's table, and AA's independent τ³-Banking run put it joint-first with Ling 3.0 Tiny at 21%, against 8% for the next model. If knowledge recall is your use case, it isn't.

← Back to index