glm: fix streaming prefill failures for real-size prompts

[andreaborio]

On the glm5.2 branch, GLM 5.2 SSD-streaming generation fails for any prompt above roughly the full-layer threshold (~64 tokens):

ds4: Metal model range 239.43..240.42 GiB is not covered by mapped model views
ds4: Metal streaming prefill batch selected addr failed to map overflow expert views at layer 9
ds4: GLM Metal prefill failed

Repro: ./ds4 --metal --ssd-streaming -m GLM-5.2-<routed-Q2K>.gguf --ctx 1024 --nothink --temp 0 -n 8 -p "<any ~300-word prompt>". Short benchmark prompts and --perplexity-file texts under the overflow point never hit it, which is probably why it survived so far.

Two root causes, one commit each:

1. The indexed prefill hardcodes full_layer_prefill=false in its per-layer mapping and forces the runtime flag off, so every chunk takes the selected-expert addr path. A chunk above ~64 tokens selects more unique experts per layer than the cache holds; the loader's overflow branch then addresses experts straight out of the mapped model, but the decode-expert-cache spans do not cover the routed expert tensors, and prefill dies. The fix computes the same threshold the plain prefill flow uses (glm_graph_stream_prefill_full_layer_enabled) and passes it to both the mapping and the runtime flag. Big chunks now stream full layers with the mm_id kernels; short prompts keep the addr path unchanged.

2. prepare_selected_batch spills to overflow views instead of evicting when the cache is at budget (force_reuse was hardcoded 0). Any small-chunk prefill against a full cache failed the same way — most visibly checkpoint-extend prefills, which broke the disk KV cache flow right after it loaded a matching checkpoint. The fix reuse-evicts at budget exactly like the decode loader does (commands are synced by the caller before prepare, so reuse cannot race the GPU, and prepare_load_buffers protects the layer's unique selected set). Overflow views remain as the last resort for allocation failure.

Testing

Machine: Apple M5 Pro 64 GB, macOS 26 (Darwin 25.5.0), Metal backend, branch = glm5.2 + these two commits, make clean && make, git diff --check clean.

Correctness track:

• ./ds4_test --server — OK
• ./ds4_test --metal-kernels — OK
• DS4_TEST_SSD_STREAMING=1 DS4_TEST_MODEL=DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf ./ds4_test --metal-ssd-streaming-cache-pressure — OK (exercises the modified loader on the DeepSeek family)
• same env, ./ds4_test --logprob-vectors — OK (official-vector gate; the shared-loader change does not perturb DeepSeek logits)

GLM 5.2 runtime (753B, routed experts Q2_K, attention Q8_0; note: my local artifact uses a ds4-native tensor layout enabled by a separate local patch — the two fixes here are layout-independent and sit in code that never inspects tensor dtypes):

• the repro above completes instead of failing; a 795-token prompt prefills at 9.4 t/s vs 1.5 t/s for the addr path on chunks that did fit before (full-layer streaming reads sequentially);
• --decode-consistency 8 stays bit-exact vs a fresh prefill (decode untouched);
• disk KV cache now works end to end: 321-token request 181 s cold, 32 s on a same-prefix follow-up (checkpoint hit, 14-token suffix prefill), 33 s across a server restart (checkpoint loaded from disk in ~16 ms).

Speed track: decode is unaffected (same path); prefill only changes for chunk sizes that previously failed outright, where full-layer streaming measures 6x faster than the (broken) addr path would need to be. One observable shift: scoring a text that stays under the overflow point moves +0.7% ppl because those chunks now take the mm_id kernels instead of addr — the same accumulation-order difference already documented in ds4_gpu_stream_prefill_batch_selected_addr_auto gating.

Found while enabling the disk KV cache for agentic use; happy to split or adjust if you prefer a different fix shape (e.g. mapping the expert spans in addr mode instead of switching to full-layer above the threshold).

[garnetlyx]

Independent confirmation and validation from a second machine.

We hit the same failure yesterday on a Mac Studio M4 Max 128 GB (macOS 26.3.1, Metal) running the published GLM-5.2-UD-Q2_K_RoutedQ2K.gguf from an external Thunderbolt 4 NVMe, and traced it independently to the same root cause as commit 1 before finding this PR: the indexed prefill flow passes a hardcoded full_layer_prefill=false to the per-layer mapper, so the decode-expert-cache spans never include the routed expert tensors. DS4_METAL_STREAMING_MAP_TRACE=1 shows every layer mapped as mode=decode-expert-cache, and the first layer whose per-chunk unique selected experts exceed the shared cache budget (layer 19 for us at a 48 GB budget, ~12–16 MoE layers in) dies in the overflow wrap with Metal model range ... is not covered by mapped model views. Forcing DS4_METAL_GLM_STREAMING_PREFILL_FULL_LAYER=1 just moves the failure to the first MoE layer, because the mapper side still installs decode spans — mapper and kernels disagree, exactly as this PR describes.

Validation of this PR's two commits applied as-is on top of glm5.2 (bd89932), make clean && make:

• make test — all suites OK (server, logprob-vectors, long-context, tool-call-quality, metal-kernels, metal-ssd-streaming-cache-pressure, local-golden-vectors, metal-short-prefill, metal-tensor-equivalence, streaming-decode-prefill-correctness; DeepSeek suites against Flash q2-imatrix).
• GLM 5.2 streamed prefill repro: 17,623-token prompt, --ssd-streaming --ssd-streaming-cache-experts 48GB --ctx 32768 --nothink --temp 0. Before: aborts at the first cache-saturated layer. After: prefill 24.31 t/s, and the model returns a faithful one-sentence summary of the document.
• Long-context: with an equivalent patch for commit 1 alone we also ran an 86,361-token prompt at --ctx 100000 (same GGUF, 48 GB cache): prefill 24.15 t/s, output starts with the expected continuation byte and shows no corruption markers. So the fix holds at ~86K context on a 128 GB box, not just at the ~800-token repro size.

We did not specifically exercise the commit-2 checkpoint-extend repro, but the cache-pressure and decode-prefill-correctness suites pass with the evict-at-budget change in place.

Thanks for the fix — we are carrying these two commits on our integration branch until this merges.