perf(qwen35): prototype concurrent library tuning

[Graffioh]

Summary

• add an opt-in step-wide logits readback for concurrent non-greedy sampling
• add an opt-in hipBLASLt plan cache keyed by exact wide Q4_K GEMM shape on RDNA 3.5
• add reproducible harness controls for temperature, seed, and both experimental paths
• preserve PR bench(concurrency): add ragged paired benchmark #596 behavior by default because neither path cleared the Strix Halo smoke-test bar

This remains a draft/experimental PR, not a measured performance win.

Stack

Depends on #596 and is based on its exact current head, 1e67ef28.

PR #596's head branch lives only in the contributor fork, so this draft targets main. Its diff collapses to the three commits in this PR once #596 lands.

Controls

Both experimental paths are disabled by default:

• DFLASH_BATCHED_SAMPLING=1 enables step-wide logits readback
• DFLASH_HIPBLASLT=1 enables the direct exact-shape plan cache
• DFLASH_HIPBLASLT_WORKSPACE_MB caps workspace (default: 32 MiB)
• DFLASH_HIPBLASLT_HEURISTICS bounds queried candidates (default: 8)
• DFLASH_HIPBLASLT_LOG=1 logs cached selections and fallbacks

The benchmark harness exposes TEMPERATURE, SEED, HIPBLASLT_MODE, and BATCHED_SAMPLING_MODE, and records the resulting server environment.

Strix Halo screening

All runs used K=8, short ragged 300–638-token prompts, one screening repeat, and exact output-token counts. Full repeats were intentionally skipped after the smoke tests did not show a credible incremental win.

Comparison	C / output	Aggregate tok/s	Output-window tok/s	Request decode tok/s	Result
direct hipBLASLt on	16 / 256	61.67	85.84	5.81	no steady-decode gain
same PR597 binary, selector off	16 / 256	61.95	84.72	5.82	control was slightly better overall
batched sampling on	8 / 128	34.82	55.47	7.04	below PR596 control
PR596 sampling control	8 / 128	35.27	56.90	7.22	+1–3% over batched path
batched sampling on	16 / 128	45.22	67.98	5.54	below PR596 aggregate/decode
PR596 sampling control	16 / 128	45.54	64.75	5.70	output-window difference was cohort noise
final PR597, both paths off	8 / 128	35.79	57.70	7.32	fallback matches PR596 envelope
llama.cpp sampling	8 / 128	22.79	32.16	4.07	Lucebox stack remains +52.8% aggregate / +72.5% decode

The hipBLASLt implementation caches the library's first ordered heuristic candidate; it does not benchmark candidates. PR #596 already routes rocBLAS through hipBLASLt on gfx1151, so the direct path duplicates essentially the same selection and is flat.

The batched readback removes per-row synchronizations, but full-vocabulary transfer barriers are not a material part of this workload. Pinned readback and parallel CPU sampling were also screened locally and removed after they failed to produce a clear request-decode gain.

Validation

• configured and built Release HIP for gfx1151 with ROCm 7.2.4
• built ggml-hip and dflash_server
• passed test_seq_engine_contract (15), test_seq_batch_plan (60), and test_seq_slot_manager (146)
• passed test_platform_compat
• smoke requests completed 16/16 at C16 and 8/8 at C8 with exact requested token counts
• bash -n on the benchmark harness
• git diff --check