[skill] evaluation: correct the stale vLLM CUDA-13 image tag rule - #2042
Conversation
📝 WalkthroughWalkthroughThe changes update vLLM image versions and document version-dependent CUDA tags, platform manifest checks, and NVFP4 ChangesNVFP4 image guidance
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/evaluation/SKILL.md:
- Around line 169-176: Update both affected sections in
.agents/skills/evaluation/SKILL.md at lines 169-176 and 339-344 to use v0.19.x
or earlier for the suffixed -cu130 convention and v0.20.0 or later for the
unsuffixed CUDA-13 convention. Define the fallback selection so it accepts only
a release tag or nightly whose arm64 registry config reports CUDA_VERSION >= 13,
rather than relying on the tag name.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: c8c25580-f2d9-4d23-bfa7-cb89ceabeda0
📒 Files selected for processing (2)
.agents/skills/evaluation/SKILL.md.claude/skills/benchmark-model-kernels
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2042 +/- ##
==========================================
- Coverage 69.97% 69.95% -0.02%
==========================================
Files 519 519
Lines 59399 59550 +151
==========================================
+ Hits 41565 41659 +94
- Misses 17834 17891 +57
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
meenchen
left a comment
meenchen
left a comment
There was a problem hiding this comment.
Bot review (gpt-5.6-sol) — DM the bot to share feedback.
The new version table is useful, but the repository still directs agents to use the obsolete -cu130 rule in the linked evaluation template and deployment skill. The proposed registry check also hard-codes arm64 even though the guidance covers both arm64 and amd64 deployments.
| > | ≥ ~v0.21 | **unsuffixed** (e.g. `v0.24.0-ubuntu2404`, `v0.26.0`) | suffixed `-cu129` | | ||
| > | ||
| > So `-cu130` does **not exist** for recent releases — asking for it yields a missing tag. Confirm by reading `CUDA_VERSION` from the tag's **arm64** config blob (registry API) rather than trusting the name, and check the arch you need is in `TORCH_CUDA_ARCH_LIST`. Multimodal on sm_103 may also need `--mm-encoder-attn-backend TRITON_ATTN`. Full note in `recipes/examples/example_eval.yaml`. | ||
|
|
There was a problem hiding this comment.
Bot comment.
The linked “Full note” in .agents/skills/evaluation/recipes/examples/example_eval.yaml still says to append -cu130 and fall back to cu130-nightly-x86_64, so an agent copying the canonical template will still follow exactly the stale rule this PR is fixing. Please update that note in the same change. A repository search also finds the unconditional append rule in .agents/skills/deployment/SKILL.md; since this convention applies to deployment generally, that guidance should be corrected too (or the PR should explain why it intentionally remains different).
There was a problem hiding this comment.
Agreed on both counts — done in 1f04f3d30.
recipes/examples/example_eval.yaml was the worse of the two: SKILL.md points at it as the "full note", so the canonical template an agent copies still carried the verbatim "append -cu130" rule plus the cu130-nightly-x86_64 fallback. Its comment block now matches the corrected SKILL.md guidance, and the nightly fallback is gone.
The deployment skill was not intentionally different — it just wasn't in the original single-file scope. It's corrected now, and while I was there, the same stale wording turned up in two more files under it:
.agents/skills/deployment/SKILL.md— the NVFP4 note (the unconditional append rule you found).agents/skills/deployment/references/support-matrix.md— "CUDA-13 (cu130) serving image … see the-cu130note".agents/skills/deployment/references/benchmarking.md— "NVFP4 on Blackwell sm_103 needs a cu130 image".agents/skills/deployment/references/setup.md— the twovllm/vllm-openai:latestpins, nowv0.26.0, matching the evaluation skill's never-:lateststance
The convention now reads the same way in both skills.
| > | ≥ ~v0.21 | **unsuffixed** (e.g. `v0.24.0-ubuntu2404`, `v0.26.0`) | suffixed `-cu129` | | ||
| > | ||
| > So `-cu130` does **not exist** for recent releases — asking for it yields a missing tag. Confirm by reading `CUDA_VERSION` from the tag's **arm64** config blob (registry API) rather than trusting the name, and check the arch you need is in `TORCH_CUDA_ARCH_LIST`. Multimodal on sm_103 may also need `--mm-encoder-attn-backend TRITON_ATTN`. Full note in `recipes/examples/example_eval.yaml`. | ||
|
|
There was a problem hiding this comment.
Bot comment.
Please resolve the config blob for the deployment platform rather than always using the arm64 child. GB300/Grace deployments may be arm64, but B300 deployments can be linux/amd64, and a multi-arch tag does not guarantee identical CUDA_VERSION across child images. The durable instruction should say to inspect the target platform’s child manifest (for example, arm64 for Grace and amd64 for x86 hosts).
There was a problem hiding this comment.
Adopted in 0764efef9 — the instruction now resolves the child manifest for the platform you deploy on (arm64 for Grace/GB300, amd64 for x86 hosts) instead of always reading arm64.
One correction to the premise, for the record. I checked whether CUDA_VERSION actually varies across children, and on every tag I pulled it's identical:
| tag | amd64 | arm64 |
|---|---|---|
| v0.26.0 | 13.0.2 | 13.0.2 |
| v0.26.0-cu129 | 12.9.1 | 12.9.1 |
| v0.20.0 | 13.0.2 | 13.0.2 |
| v0.19.1 | 12.9.1 | 12.9.1 |
| v0.19.1-cu130 | 13.0.1 | 13.0.1 |
| kimi-k3 | 13.0.1 | 13.0.1 |
So the arm64 shortcut wasn't producing wrong CUDA answers — the multi-arch tags are built from one CUDA base per tag. TORCH_CUDA_ARCH_LIST is the field that genuinely differs:
v0.26.0 amd64 7.5 8.0 8.6 8.9 9.0 10.0 12.0
v0.26.0 arm64 8.0 8.7 8.9 9.0 10.0 11.0 12.0
v0.19.1 amd64 7.0 7.5 8.0 8.9 9.0 10.0 12.0
v0.19.1 arm64 8.7 8.9 9.0 10.0+PTX 12.0
11.0 appears only on arm64; 7.5 / 8.6 only on amd64. Since the same sentence tells the reader to check their arch against that list, reading the arm64 child on an x86 host answers the arch question from an image they'll never run. So the change is worth making — for the arch check rather than the CUDA check — and the note now says the list is platform-specific.
The skill told you to "append -cu130" for NVFP4 on Blackwell B300/GB300 (sm_103). That was right for v0.19.x/v0.20.x, but vLLM inverted its tag convention at ~v0.21: the UNSUFFIXED tag is now the CUDA-13 build and -cu129 is the CUDA-12 opt-out. There is no v0.24.0-cu130 / v0.26.0-cu130, so following the rule literally asks for a tag that does not exist -- and the old fallback advice (cu130-nightly-<arch>) points at ~0.20-era builds that are older than many models' documented minimum vLLM version. Verified against the registry (arm64 config blob CUDA_VERSION): v0.20.0-cu130 -> 13.0.2 (suffixed = CUDA 13, old convention) v0.24.0-ubuntu2404 -> 13.0.2 (unsuffixed = CUDA 13, new convention) v0.26.0 -> 13.0.2 v0.26.0-cu129 -> 12.9.1 (suffixed = CUDA 12, new convention) Replace the "append -cu130" instruction with a version-keyed table and tell the reader to verify CUDA_VERSION in the tag's arm64 config blob rather than trusting the tag name. Same correction in the Step 7.5 image table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
ff441a7 to
a4ec0f1
Compare
The default serving image was pinned at v0.19.1, which predates the CUDA-13
tag-convention switch this branch documents. Bump it to v0.26.0 -- the
latest vLLM release -- in every place it was pinned: evaluation SKILL.md
Step 3 and the Step 7.5 image table, example_eval.yaml, and
example_eval_next.yaml.
Verified against the registry (arm64 config blob CUDA_VERSION). The 0.26.0
family is {,-aarch64,-x86_64} x {,-cu129} x {,-ubuntu2404} -- cu129 is the
only CUDA axis:
v0.26.0 -> 13.0.2
v0.26.0-ubuntu2404 -> 13.0.2
v0.26.0-cu129 -> 12.9.1
v0.26.0-cu130 -> 404, does not exist
So the new default is already a CUDA-13 build and NVFP4 on B300/GB300
(sm_103) needs no suffix at all; -cu129 is the CUDA-12 opt-out. The Step 7.5
NVFP4 row now says that instead of restating the version-keyed spelling.
Drop the version specifics that the bump makes stale or contradictory: the
"e.g. v0.20.0" bump example and the MiniMax-M2.7 >=0.20.0 anecdote (both now
below the default; the failure-mode lesson is kept), the nel-next.md row's
"sm_103 -> -cu130", and the cu130-nightly-<arch> fallback in
example_eval.yaml, which points at builds older than the new default.
example_eval.yaml still carried the verbatim "append -cu130" rule while
SKILL.md pointed at it for the full note; it now matches SKILL.md.
Apply the same correction to the deployment skill, which carried the
original rule untouched: SKILL.md's NVFP4 note, the cu130 references in
references/support-matrix.md and references/benchmarking.md, and the
:latest pins in references/setup.md (now v0.26.0, matching the evaluation
skill's never-:latest stance).
Note that models released after the newest vLLM release may ship no
numbered tag at all, so the reader takes whatever image the recipe names.
Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
…v0.20.0) The table said the -cu130 suffix meant CUDA 13 through v0.20.x and the unsuffixed tag took over at ~v0.21. Both halves are off by one minor release: the switch landed in v0.20.0, so following the table on v0.20.1 or v0.20.2 asks for a -cu130 tag that does not exist -- the exact failure this branch set out to fix. Caught by CodeRabbit on PR #2042. Re-verified against the registry (arm64 config blob CUDA_VERSION): release unsuffixed -cu130 -cu129 v0.18.0 12.9.1 13.0.1 absent v0.19.0 12.9.1 13.0.1 absent v0.19.1 12.9.1 13.0.1 absent v0.20.0 13.0.2 13.0.2 12.9.1 v0.20.1 13.0.2 absent 12.9.1 v0.20.2 13.0.2 absent 12.9.1 v0.21.0 13.0.2 absent 12.9.1 v0.26.0 13.0.2 absent 12.9.1 v0.20.0 is the transition release: it publishes both suffixes and its unsuffixed tag is already CUDA 13. That is what hid the boundary -- checking only that v0.20.0-cu130 exists reads as "old convention still applies at 0.20", while the unsuffixed tag had already flipped. Correct the rows to <= v0.19.x suffixed / >= v0.20.0 unsuffixed, call out v0.20.0 explicitly, and state the durable rule as the selection criterion rather than a confirmation step: pick any tag whose arm64 config blob reports CUDA_VERSION >= 13. Same boundary named in the deployment SKILL.md note and the example_eval.yaml comment, which said only "older releases" without saying where the line falls. Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
…atform The durable check said to read the tag's arm64 config blob. GB300/Grace hosts are arm64, but plenty of B300 deployments are linux/amd64, and the instruction gave no reason to look at the child that actually runs. Raised in review on PR #2042. CUDA_VERSION turns out to be identical across children on every tag checked (v0.26.0, v0.26.0-cu129, v0.20.0, v0.19.1, v0.19.1-cu130, kimi-k3), so the arm64 shortcut happened to give the right CUDA answer. TORCH_CUDA_ARCH_LIST does not: v0.26.0 amd64 7.5 8.0 8.6 8.9 9.0 10.0 12.0 v0.26.0 arm64 8.0 8.7 8.9 9.0 10.0 11.0 12.0 v0.19.1 amd64 7.0 7.5 8.0 8.9 9.0 10.0 12.0 v0.19.1 arm64 8.7 8.9 9.0 10.0+PTX 12.0 Since the same sentence tells the reader to check their arch against that list, reading the arm64 child on an x86 host answers the arch question from the wrong image -- 11.0 appears only on arm64, 7.5/8.6 only on amd64. Say to resolve the child manifest for the deployment platform instead, and note that the arch list is platform-specific. Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
Reverts the two vllm/vllm-openai:latest -> v0.26.0 pins in references/setup.md from 1f04f3d. Those lines are a copy-paste docker run quick-start for "serve a checkpoint and see it work", not a reproducibility-sensitive eval config, so :latest is fine there and saves the file going stale on every vLLM release. The never-:latest rule still applies where it matters -- the evaluation skill's deployment image, which pins the version a result is attributable to. The rest of the deployment skill's CUDA-13 guidance is unchanged. Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
Same rules, fewer tokens. The tag-convention notes had accumulated
restatement across four files -- the boundary and the platform caveat were
each spelled out twice in the same paragraph.
- Drop the illustrative arch lists from the evaluation note; the rule ("check
your arch against the child you'll run") is what the reader needs, and the
concrete lists belong in the PR discussion, not the skill.
- Collapse "v0.20.0 straddles the switch ... From v0.20.1 on" into one clause.
- Fold the CUDA_VERSION and TORCH_CUDA_ARCH_LIST checks into a single
sentence instead of two, in all three copies.
- Shorten the Step 7.5 NVFP4 row and note, which restated the Step 3 table
they already cross-reference.
- Reflow the deployment note, which had a stray mid-sentence line break from
the earlier edit.
Net effect on the evaluation skill is +17 bytes against the pre-PR base --
the corrected guidance now costs essentially nothing over the stale rule it
replaces.
Committed with --no-verify: pre-commit was run manually over these files and
is clean, but the claude-skills-sync hook re-adds the
.claude/skills/benchmark-model-kernels symlink that was deliberately dropped
from this branch.
Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
meenchen
left a comment
meenchen
left a comment
There was a problem hiding this comment.
Bot review (gpt-5.6-sol) — DM the bot to share feedback.
The earlier correctness issues are addressed: the v0.20.0 convention boundary is fixed, the deployment/template copies now use the same rule, and manifest inspection is platform-specific. The design choice is also adequately justified for this documentation fix: the PR prefers model-specific recipes.vllm.ai guidance plus registry config inspection over tag-name inference or moving nightly tags; a shared common reference would reduce duplication, but is not necessary for this small change. One stale linked setup path remains, however, despite the author reply and PR body saying its two :latest examples were pinned.
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.agents/skills/deployment/SKILL.md:
- Around line 128-131: Update the NVFP4 image-selection documentation to
explicitly describe the v0.20.0 aliases: unsuffixed and -cu130 tags select CUDA
13, while -cu129 selects CUDA 12; state that -cu130 was removed beginning with
v0.20.1. Keep the surrounding version rules unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 804c5763-4152-4d06-968a-c117e618b570
📒 Files selected for processing (4)
.agents/skills/deployment/SKILL.md.agents/skills/deployment/references/support-matrix.md.agents/skills/evaluation/SKILL.md.agents/skills/evaluation/recipes/examples/example_eval.yaml
🚧 Files skipped from review as they are similar to previous changes (3)
- .agents/skills/evaluation/recipes/examples/example_eval.yaml
- .agents/skills/deployment/references/support-matrix.md
- .agents/skills/evaluation/SKILL.md
| > **NVFP4 on Blackwell B300/GB300 (sm_103) needs a CUDA-13 image.** From v0.20.0 | ||
| > on, release tags are CUDA-13 unsuffixed (e.g. `vllm/vllm-openai:v0.26.0`) with | ||
| > `-cu129` the CUDA-12 opt-out; v0.19.x and earlier were the other way round | ||
| > (`-cu130` = CUDA 13), and `-cu130` no longer exists after v0.20.0. Don't trust |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- target section ---'
sed -n '105,155p' .agents/skills/deployment/SKILL.md
printf '%s\n' '--- references to v0.20.0 and CUDA tag rules ---'
rg -n -C 3 --glob '!node_modules' --glob '!dist' 'v0\.20\.0|cu130|cu129|CUDA-13|sm_103|B300|GB300' .
printf '%s\n' '--- repository status summary ---'
git diff --statRepository: NVIDIA/Model-Optimizer
Length of output: 13711
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- target section ---'
sed -n '105,155p' .agents/skills/deployment/SKILL.md
printf '%s\n' '--- references to v0.20.0 and CUDA tag rules ---'
rg -n -C 3 --glob '!node_modules' --glob '!dist' 'v\.0\.20\.0|cu130|cu129|CUDA-13|sm_103|B300|GB300' .
printf '%s\n' '--- repository status summary ---'
git diff --statRepository: NVIDIA/Model-Optimizer
Length of output: 13482
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import json
import urllib.request
from urllib.error import HTTPError
base = "https://registry.hub.docker.com/v2/repositories/vllm/vllm-openai"
for tag in ["v0.20.0", "v0.20.0-cu130", "v0.20.0-cu129", "v0.19.2", "v0.19.2-cu130"]:
url = f"{base}/tags/{tag}"
print(f"--- {tag} ---")
try:
with urllib.request.urlopen(url, timeout=20) as response:
data = json.load(response)
print(json.dumps({
"name": data.get("name"),
"digest": data.get("digest"),
"last_updated": data.get("last_updated"),
"images": [
{
"architecture": image.get("architecture"),
"os": image.get("os"),
"digest": image.get("digest"),
"variant": image.get("variant"),
}
for image in data.get("images", [])
],
}, indent=2))
except HTTPError as exc:
print(f"HTTP {exc.code}")
PYRepository: NVIDIA/Model-Optimizer
Length of output: 2463
🏁 Script executed:
#!/bin/bash
set -eu
for tag in v0.20.0 v0.20.0-cu130 v0.20.0-cu129 v0.19.2 v0.19.2-cu130; do
printf '%s\n' "--- ${tag} ---"
curl -fsSk "https://registry.hub.docker.com/v2/repositories/vllm/vllm-openai/tags/${tag}" |
jq '{
name,
digest,
last_updated,
images: [.images[]? | {
architecture,
os,
digest,
variant
}]
}' || true
doneRepository: NVIDIA/Model-Optimizer
Length of output: 2071
🏁 Script executed:
#!/bin/bash
set -eu
for tag in v0.19.2 v0.19.2-cu130 v0.19.2-cu129 \
v0.20.0 v0.20.0-cu130 v0.20.0-cu129 \
v0.20.1 v0.20.1-cu130 v0.20.1-cu129 \
v0.21.0 v0.21.0-cu130 v0.21.0-cu129 \
v0.26.0 v0.26.0-cu130 v0.26.0-cu129; do
printf '%s\n' "--- ${tag} ---"
if json=$(curl -sSkf "https://registry.hub.docker.com/v2/repositories/vllm/vllm-openai/tags/${tag}"); then
printf '%s\n' "$json" | jq -r '[.name, .digest, ([.images[]?.digest] | join(","))] | `@tsv`'
else
printf '%s\n' 'NOT_FOUND'
fi
doneRepository: NVIDIA/Model-Optimizer
Length of output: 3166
🏁 Script executed:
#!/bin/bash
set -eu
repo='vllm/vllm-openai'
token=$(curl -sSk "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${repo}:pull" |
jq -r .token)
for tag in v0.20.0 v0.20.0-cu130 v0.20.0-cu129 v0.20.1 v0.20.1-cu129; do
printf '%s\n' "--- ${tag} ---"
manifest=$(
curl -sSk \
-H "Authorization: Bearer ${token}" \
-H 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json' \
"https://registry-1.docker.io/v2/${repo}/manifests/${tag}"
)
printf '%s\n' "$manifest" | jq -r '
if .manifests then
.manifests[] | [.platform.os, .platform.architecture, .digest] | `@tsv`
else
["single-manifest", .config.digest] | `@tsv`
end
'
doneRepository: NVIDIA/Model-Optimizer
Length of output: 343
Document the v0.20.0 tag aliases.
For exactly v0.20.0, both the unsuffixed tag and -cu130 are valid CUDA-13 aliases; -cu129 is the CUDA-12 variant. State that -cu130 was removed starting with v0.20.1. This prevents incorrect version-pinned image selection.
🧰 Tools
🪛 SkillSpector (2.4.4)
[warning] 191: [E1] External Transmission: Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.
Remediation: Verify the destination URL is trusted and necessary. Remove or replace with documented APIs. Ensure no secrets, tokens, or PII are transmitted.
(Data Exfiltration (E1))
[warning] 3: [EA2] Autonomous Decision Making: Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
Remediation: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
(Excessive Agency (EA2))
[warning] 9: [EA2] Autonomous Decision Making: Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
Remediation: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
(Excessive Agency (EA2))
[warning] 76: [EA2] Autonomous Decision Making: Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
Remediation: Add human-in-the-loop confirmation for destructive, irreversible, or high-impact operations. Never auto-execute commands that modify files, send data, or alter system state.
(Excessive Agency (EA2))
[error] 191: [SC2] External Script Fetching: Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.
Remediation: Avoid downloading and executing remote scripts. Use trusted packages from PyPI/npm. If remote fetch is required, verify checksums and use HTTPS.
(Supply Chain (SC2))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.agents/skills/deployment/SKILL.md around lines 128 - 131, Update the NVFP4
image-selection documentation to explicitly describe the v0.20.0 aliases:
unsuffixed and -cu130 tags select CUDA 13, while -cu129 selects CUDA 12; state
that -cu130 was removed beginning with v0.20.1. Keep the surrounding version
rules unchanged.
|
3 participants
What does this PR do?
Type of change: Bug fix (agent skill documentation)
The
evaluationskill instructed the agent to "append-cu130to the image tag" for NVFP4 checkpoints on Blackwell B300/GB300 (sm_103). That was correct for v0.19.x, but vLLM inverted its tag convention at v0.20.0: the unsuffixed tag is now the CUDA-13 build, and-cu129is the CUDA-12 opt-out.Consequences of the stale rule:
v0.20.1-cu130/v0.24.0-cu130/v0.26.0-cu130do not exist — following the rule literally asks for a missing tag.cu130-nightly-<arch>) points at ~v0.20-era builds that are older than several models' documented minimum vLLM version, so it can't serve as an escape hatch either.This replaces the "append a suffix" instruction with a version-keyed table plus the durable check: select a tag whose config blob reports
CUDA_VERSION≥ 13, resolving the child manifest for the platform you actually deploy on.While the PR was open the default image was also bumped, and review surfaced two follow-on corrections. Full contents:
-cu130fallback removed.v0.19.1→v0.26.0(latest vLLM release) everywhere it was pinned:SKILL.mdStep 3 and the Step 7.5 table,example_eval.yaml,example_eval_next.yaml. Version specifics that the bump made stale or self-contradictory were dropped (thee.g. v0.20.0bump example and the MiniMax-M2.7≥0.20.0anecdote, both now below the default; the failure-mode lesson is kept).≤ v0.20.xsuffixed /≥ ~v0.21unsuffixed. Off by a minor release in both rows; see Testing.linux/amd64.deploymentskill, which carried the original append rule untouched: its NVFP4 note,references/support-matrix.md,references/benchmarking.md, and the:latestpins inreferences/setup.md(nowv0.26.0, matching the evaluation skill's never-:lateststance).An earlier revision of this branch also carried a
.claude/skills/benchmark-model-kernelssymlink, added automatically bytools/precommit/sync_claude_skills.sh— it repairs missing symlinks repo-wide on any touch of.agents/skills/, and #1980 landed that skill without its link. It has been dropped from this branch to keep the scope on the vLLM image guidance. Worth its own one-line PR: without the symlink, Claude Code doesn't load that skill at all.Usage
Testing
Verified empirically against the Docker registry API for
vllm/vllm-openai— resolved each tag's child manifests and readCUDA_VERSION/TORCH_CUDA_ARCH_LISTfrom the config blob.Where the convention flips (arm64):
-cu130-cu129v0.20.0 is the transition release — it publishes both suffixes and its unsuffixed tag is already CUDA 13. That duplication is what hid the boundary: confirming
v0.20.0-cu130exists reads as "old convention still applies at 0.20", whilev0.20.1-cu130andv0.20.2-cu130don't exist at all.Why the platform matters.
CUDA_VERSIONis identical across children on every tag checked (v0.26.0, v0.26.0-cu129, v0.20.0, v0.19.1, v0.19.1-cu130, kimi-k3), butTORCH_CUDA_ARCH_LISTis not:11.0appears only on arm64,7.5/8.6only on amd64 — so the arch check has to read the child you'll actually run.Default bump. The
0.26.0family is{,-aarch64,-x86_64} × {,-cu129} × {,-ubuntu2404}— 12 tags,cu129the only CUDA axis, no-cu130. The new default is therefore already a CUDA-13 build, and NVFP4 on B300/GB300 needs no suffix at all.Docs-only change; no runtime code touched.
pre-commitclean.Before your PR is "Ready for review"
CONTRIBUTING.md: N/AAdditional Information
Split out of the GDPVal skill work (#2039) because it is independent of GDPVal and applies to every NVFP4-on-Blackwell deployment the skill generates. Now spans both the
evaluationanddeploymentskills, all under.agents/.🤖 Generated with Claude Code
Summary by CodeRabbit