recipe(lt-wikidata-comp-zh): add feature extraction recipes

[kujin66]

Summary

Adds nested CPU fp32 and fp16 feature-extraction recipes for dell-research-harvard/lt-wikidata-comp-zh, a Chinese BERT checkpoint used for sentence/linking representations. This is an Effort L0 / Outcome L0 recipe-only contribution; both required tuples reached L3 with matching pinned Chinese sentence-similarity metrics.

Model metadata

What the model does

The pinned checkpoint emits contextual token embeddings (last_hidden_state) for Chinese text. Sentence-level similarity uses attention-mask-aware mean pooling outside the exported BertModel graph. Confidence: verified from the pinned checkpoint config/model card, Transformers BertModel, exported ONNX interface, and evaluation behavior.

Primary user stories

• Produce contextual Chinese token representations for feature-extraction consumers.
• Produce sentence embeddings for Chinese semantic similarity/linking by applying mask-aware mean pooling in the evaluator or application.

Supported tasks

• WinML recipe/export task: feature-extraction (AutoModel / BertModel).
• Checkpoint pipeline tag: sentence-similarity; downstream pooling is intentionally outside ONNX.
• Pinned checkpoint: dell-research-harvard/lt-wikidata-comp-zh@c2b245220e8cd6db8777afeb4864412e6d7dae3a.

Model architecture

BertModel (102,267,648 parameters)
├── Embeddings (vocab 21,128 × hidden 768; max sequence 512)
├── Encoder stack × 12
│   ├── Self-attention (12 heads; hidden 768)
│   ├── Feed-forward (768 → 3,072 → 768, GELU)
│   └── Residual + LayerNorm
└── last_hidden_state [batch, sequence, 768]
    └── downstream mask-aware mean pooling (outside exported graph)

• Source/confidence: pinned checkpoint config/model card, Transformers BertModel, HTP metadata, and ONNX graph (verified).

Validation and support evidence

Baseline

• Current main: 5deebd422e95f28fe8fd912ee50ce874710187b3; WinML CLI 0.2.0.
• Recipe-free current-main fp32 export: artifact completed in 44.0 s; 102.3M parameters, 228 modules, 88 traced modules, opset 17.
• Auto-config already selected AutoModel, BertIOConfig, WinMLModelForFeatureExtraction, fp32 (quant: null), and optim.clamp_constant_values: true.
• Optimum probe: feature-extraction was vendor-registered before WinML registration; WinML added no task registration. This is recipe coverage, not a class-wide source fix.
• CPU fp32 baseline: mean 238.386 ms, p50 238.832 ms, 4.19 samples/s, +436.87 MB RSS.
• Pinned evaluation (mteb/sts22-crosslingual-sts@91d97a5b9d761e285ac3e1b4f239797bbd21c4b8, zh, test, first 20 streaming examples, no shuffle): cosine Spearman 75.4846.
• Build/eval/perf output artifacts completed, but each CLI process subsequently returned Windows native exit -1073741819 after OpenVINO plugin registration reported missing openvino.dll. Results below preserve that host teardown condition rather than claiming a clean process exit.

Goal

• Effort: L0 — nested recipes only; no production source changes.
• Goal ceiling: L3 — both required CPU precision tuples must build, run with concrete perf, pass PyTorch/ONNX parity, and return a pinned task metric.
• Outcome target: L0 — recipe coverage for support already present on current main.

Outcome

• Outcome: L0, highest reached goal: L3 for CPU fp32 and fp16.
• Required coverage: 2/2 tuples validated; no deferred tuples.
• Shipped paths:
  • examples/recipes/dell-research-harvard_lt-wikidata-comp-zh/cpu/cpu/feature-extraction_fp32_config.json
  • examples/recipes/dell-research-harvard_lt-wikidata-comp-zh/cpu/cpu/feature-extraction_fp16_config.json
• No production code or recipe README changes. No new model-family or methodology finding was needed.

Per-EP/device/precision results — including perf and eval data

Tier	EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM Δ	Task metric
L0	CPUExecutionProvider / cpu	fp32	PASS (artifact; native teardown noted above)	—	—	—	—	—
L1	CPUExecutionProvider / cpu	fp32	PASS	225.059 ms	225.563 ms	4.44 samples/s	+436.78 MB	—
L2	CPUExecutionProvider / cpu	fp32	PASS	—	—	—	—	pooled cosine ≥0.999999999999; pooled max abs ≤4.411e-6; sentence-similarity max abs 3.241e-7
L3	CPUExecutionProvider / cpu	fp32	PASS	—	—	—	—	cosine Spearman 75.4846
L0	CPUExecutionProvider / cpu	fp16	PASS (artifact; native teardown noted above)	—	—	—	—	—
L1	CPUExecutionProvider / cpu	fp16	PASS	266.876 ms	267.740 ms	3.75 samples/s	+455.08 MB	—
L2	CPUExecutionProvider / cpu	fp16	PASS	—	—	—	—	pooled cosine ≥0.999999065573; pooled max abs ≤0.002968; sentence-similarity max abs 4.543e-5
L3	CPUExecutionProvider / cpu	fp16	PASS	—	—	—	—	cosine Spearman 75.4846

Perf used the same pinned named Chinese NPZ input, 5 warmups, and 30 measured iterations. fp16 is structurally genuine: 200 FLOAT16 initializers and 203,354,112 bytes external data versus fp32's 200 FLOAT initializers and 406,708,224 bytes.

Delta

Relative to the current-main auto-config:

• fp32: /eval added with the pinned Chinese STS22 dataset/config/revision, deterministic 20-example sample, streaming, and explicit sentence/score columns; all export/optimization/loader fields are identical.
• fp16: the identical /eval addition plus /quant changed from null to the existing fp16 conversion configuration (mode: fp16, fp32 I/O retained).
• Replaced flat recipes with the required nested CPU layout. The newly contributed flat w8a16 recipe was reduced to the mandatory CPU fp32 tuple; no unsupported or unrequired tuple remains.
• examples/recipes/README.md remains untouched.

Analyze summary — component level and op level

Static all-rule-backed-EP analysis completed for both artifacts; it is compatibility analysis, not runtime execution. Analyzer outputs were saved before the same host-native teardown condition noted above.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP findings
fp32	embeddings; 12× encoder attention/FFN/output; mask/shape/fused functional region	384 mapped, 0 partial, 0 unmapped; retained module scopes + explicit functional bucket	None from rule-backed EPs
fp16	embeddings; 12× encoder attention/FFN/output; mask/shape/fused functional region	385 mapped, 0 partial, 0 unmapped; retained module scopes + explicit functional bucket	None from rule-backed EPs

Op-level summary

Artifact	Graph	Dominant ops	EP roll-up
fp32	384 ops / 15 types	Reshape 120; Gemm 72; Transpose 48; Add 38; LayerNormalization 25	TensorRT RTX, QNN GPU/NPU, OpenVINO CPU/GPU/NPU: all 15 types supported; no partial/unsupported types
fp16	385 ops / 15 types	Reshape 120; Gemm 72; Transpose 48; Add 38; LayerNormalization 25	TensorRT RTX, QNN GPU/NPU, OpenVINO CPU/GPU/NPU: all 15 types supported; no partial/unsupported types

Rule-less CUDA, MIGraphX, DML, CPU, and VitisAI rows classify all 15 operator types as unknown. The detected GeluPattern (12 instances) is also unknown as a subgraph pattern; its constituent Gelu operator is supported by each rule-backed EP above. Actual CPU runtime is independently demonstrated by perf, parity, and evaluation.

Reproduce commands

$MODEL = 'dell-research-harvard/lt-wikidata-comp-zh'
$ROOT = 'examples/recipes/dell-research-harvard_lt-wikidata-comp-zh/cpu/cpu'
$RULES = '<path-to-ModelKitArtifacts>/rules'
$INPUT = '<semantic-inputs.npz>'
$DATASET_REV = '91d97a5b9d761e285ac3e1b4f239797bbd21c4b8'

winml build -m $MODEL -c "$ROOT/feature-extraction_fp32_config.json" -o '<out-fp32>' --ep cpu --rebuild --no-color
winml build -m $MODEL -c "$ROOT/feature-extraction_fp16_config.json" -o '<out-fp16>' --ep cpu --rebuild --no-color
winml perf -m '<out-fp32>/model.onnx' --ep cpu --device cpu --precision fp32 --input-data $INPUT --iterations 30 --warmup 5 --skip-build
winml perf -m '<out-fp16>/model.onnx' --ep cpu --device cpu --precision fp16 --input-data $INPUT --iterations 30 --warmup 5 --skip-build
winml eval -m '<out-fp32>/model.onnx' --model-id $MODEL --task feature-extraction --ep cpu --device cpu --dataset mteb/sts22-crosslingual-sts --dataset-name zh --dataset-revision $DATASET_REV --split test --samples 20 --no-shuffle --streaming --column input_column_1=sentence1 --column input_column_2=sentence2 --column score_column=score --skip-build
winml eval -m '<out-fp16>/model.onnx' --model-id $MODEL --task feature-extraction --ep cpu --device cpu --dataset mteb/sts22-crosslingual-sts --dataset-name zh --dataset-revision $DATASET_REV --split test --samples 20 --no-shuffle --streaming --column input_column_1=sentence1 --column input_column_2=sentence2 --column score_column=score --skip-build
$env:WINMLCLI_RULES_DIR = $RULES
winml analyze -m '<out-fp32>/model.onnx' --ep all --device all --htp-metadata '<out-fp32>/export_htp_metadata.json'
winml analyze -m '<out-fp16>/model.onnx' --ep all --device all --htp-metadata '<out-fp16>/export_htp_metadata.json'
uv run ruff check src/ tests/
uv run mypy -p winml.modelkit
uv run pytest tests/unit/models tests/unit/loader tests/unit/datasets tests/unit/export --tb=short --no-cov -m "not e2e and not npu and not gpu"

[kujin66]

Reviewer-style verification (not a formal GitHub approval; same account authored the PR):

• PR branch head: 937b000e891240234ec0ecfb0856e3501b486630.
• Scope check: diff is limited to examples/recipes/README.md plus the two examples/recipes/dell-research-harvard_lt-wikidata-comp-zh/*.json recipe files. No source, tests, or skill files are included.
• Baseline freshness: PR body cites origin/main/baseline HEAD 904c20109ab31d737c5132c3686ab404236edd57; current origin/main was refreshed and matches that commit.
• Recipe check: feature-extraction_fp16_config.json now uses quant.mode=fp16; rebuilt artifact temp/lt_wikidata_comp_zh_fp16_recipe_fixed/model.onnx has 202 FLOAT16 initializers and preserved I/O. feature-extraction_w8a16_config.json artifact contains QDQ nodes (QuantizeLinear=428, DequantizeLinear=630).
• L0/L1 evidence: PR body includes build output dirs, build-complete lines, structural I/O shapes, CPU provider snapshot, and w8a16 perf numbers.
• CI: all checks are passing at time of comment: license/cla, lint, Analyze (Python), CodeQL, and all test jobs.

Remaining gate: external maintainer/reviewer approval and conversion from Draft to Ready.

[ssss141414]

Independent final-head review: APPROVE. Verified exact head e1e9bbb from a fresh detached checkout; contributor authorship is preserved; scope is exactly the two nested CPU fp32/fp16 recipes with no README/source changes; pinned build, perf, L2 parity, eval, component/op analysis, and workflow-parity quality evidence agree with the PR body; all 9 final-head checks are COMPLETED/SUCCESS; no review threads remain.