ci: nudge multi-objective skill to trigger eval#1392
Merged
Conversation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collaborator
Author
|
/nvskills-ci |
rgsl888prabhu
added
non-breaking
Collaborator
Author
|
@cafzal I have created this PR for benchamak and skill card |
📝 WalkthroughWalkthroughAdds NVSkills benchmark and skill-card documentation, inserts a single blank line after SKILL.md front matter, and adds a DSSE/in-toto signature bundle file for the cuopt-multi-objective-exploration skill. ChangesSkill documentation and signature artifacts
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/cuopt-multi-objective-exploration/SKILL.md`:
- Line 20: The commit f0ad3b7218abb88f84b882da7fcd016e76775994 is missing the
Signed-off-by trailer; amend that commit (and any other commits in this branch
affecting **/*) by running git commit --amend -s for the most recent commit or
use git rebase -i to edit older commits, add the Signed-off-by line, then
force-push the branch (git push --force) to update the PR.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d9f4dc94-8842-4d17-97ea-25d867342dfe
📒 Files selected for processing (1)
skills/cuopt-multi-objective-exploration/SKILL.md
|
|
||
| # Multi-Objective Exploration | ||
|
|
||
|
|
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Check DCO sign-off trailers on PR commits =="
git log --format='%H%n%B%n----' | awk '
BEGIN{ok=1; RS="----\n"}
{
if ($0 ~ /^[0-9a-f]{40}\n/ && $0 !~ /Signed-off-by:[[:space:]]+/) {
split($0, a, "\n"); print "Missing Signed-off-by on commit: " a[1]; ok=0
}
}
END{ if (ok) print "All commits include Signed-off-by trailer." ; else exit 1 }'
echo
echo "== Check PR comments for /nvskills-ci trigger =="
# Requires gh auth in the verification environment
gh pr view 1392 --repo NVIDIA/cuopt --comments --json comments \
--jq '.comments[].body' | rg -n '/nvskills-ci' || {
echo "No /nvskills-ci comment found on PR `#1392`"; exit 1;
}Repository: NVIDIA/cuopt
Length of output: 178
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Check PR comments for /nvskills-ci trigger =="
# Use gh to check PR comments for NVSkills CI validation
gh pr view 1392 --repo NVIDIA/cuopt --json comments --jq '.comments[].body' | grep -c '/nvskills-ci' > /dev/null && {
echo "/nvskills-ci validation comment found"
} || {
echo "No /nvskills-ci validation comment found on PR"
}Repository: NVIDIA/cuopt
Length of output: 143
Add DCO sign-off to commits before merging.
Commit f0ad3b7 is missing the required Signed-off-by trailer. Amend all commits with git commit --amend -s and force-push to satisfy DCO policy for **/* files.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@skills/cuopt-multi-objective-exploration/SKILL.md` at line 20, The commit
f0ad3b7218abb88f84b882da7fcd016e76775994 is missing the Signed-off-by trailer;
amend that commit (and any other commits in this branch affecting **/*) by
running git commit --amend -s for the most recent commit or use git rebase -i to
edit older commits, add the Signed-off-by line, then force-push the branch (git
push --force) to update the PR.
Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>
Collaborator
Author
|
/ok to test d6eccab |
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/cuopt-multi-objective-exploration/skill-card.md`:
- Line 81: Replace the ambiguous link text "here" with descriptive link text
that explains the destination and purpose (e.g., "report NVIDIA AI concerns on
Intigriti" or "report vulnerabilities on Intigriti"), updating the markdown in
skill-card.md where the current sentence contains "Please report quality, risk,
security vulnerabilities or NVIDIA AI Concerns
[here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail)". Ensure the
link text is meaningful for screen readers and satisfies MD059 by describing the
link target and context.
- Around line 60-69: The markdown table under the "## Evaluation Results:"
header violates MD058 by not having blank lines surrounding it; fix by inserting
a blank line immediately after the "## Evaluation Results:" line and another
blank line immediately before the "## Testing Completed:" header so the table
(the rows starting with "| Dimension | Num | `claude-code` | `codex` |" through
the final efficiency row) is separated by blank lines from surrounding content.
- Line 1: The PR lacks a DCO sign-off on commit f0ad3b7 and shows no NVSkills CI
validation for changes under skills/**; fix by amending the commit message for
f0ad3b7 to add a "Signed-off-by: Your Name <you@domain>" line (git commit
--amend --no-edit then git push --force) so the DCO check can pass, and ensure
the NVSkills CI runs against the PR head SHA by re-triggering the /nvskills-ci
workflow (or creating a new empty commit/force-push) so the
NVSkills/skills-related check-runs appear on the PR; verify the check-runs
complete successfully before marking resolved.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: dd812448-0191-4ec5-99f7-7276b5e71964
📒 Files selected for processing (3)
skills/cuopt-multi-objective-exploration/BENCHMARK.mdskills/cuopt-multi-objective-exploration/skill-card.mdskills/cuopt-multi-objective-exploration/skill.oms.sig
✅ Files skipped from review due to trivial changes (1)
- skills/cuopt-multi-objective-exploration/BENCHMARK.md
| @@ -0,0 +1,81 @@ | |||
| ## Description: <br> | |||
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
gh pr view "$PR" --repo "$REPO" --json comments,commits,statusCheckRollup \
| jq -r '
def maintainer: .=="OWNER" or .=="MEMBER" or .=="COLLABORATOR";
"Maintainer /nvskills-ci comments:",
(
[.comments[]?
| select((.body | test("^/nvskills-ci\\b")) and (.authorAssociation | maintainer))
| (.author.login + " [" + .authorAssociation + "]")]
| if length==0 then "NONE" else .[] end
),
"",
"Commits missing DCO Signed-off-by:",
(
[.commits[]?
| select(((.messageBody // "") | test("(?m)^Signed-off-by:\\s+.+<.+>$")) | not)
| .oid[0:7]]
| if length==0 then "NONE" else .[] end
),
"",
"Status checks:",
(.statusCheckRollup[]? | (.context + ": " + (.state // "UNKNOWN")))
'Repository: NVIDIA/cuopt
Length of output: 446
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
# Get full-ish objects for status checks (trim fields to keep output small)
gh pr view "$PR" --repo "$REPO" --json statusCheckRollup \
| jq -r '
"statusCheckRollup_count=" + ((.statusCheckRollup|length|tostring)),
(.statusCheckRollup[]?
| {context:(.context//null), state:(.state//null), description:(.description//null), detailsUrl:(.detailsUrl//null), checkSuite:(.checkSuiteId//null), checkRun:(.checkRunId//null)}
| tojson
)
'Repository: NVIDIA/cuopt
Length of output: 3852
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
# Check for /nvskills-ci comments with authorAssociation and exact body text
gh pr view "$PR" --repo "$REPO" --json comments \
| jq -r '
"nvskills-ci comments:",
([.comments[]?
| select(.body|test("^/nvskills-ci\\b"))
| {login:.author.login, assoc:.authorAssociation, body:.body}]
| sort_by(.login)
| .[]
] | .)
'Repository: NVIDIA/cuopt
Length of output: 194
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
# Validate which commit(s) are missing DCO and show their Signed-off-by lines presence
gh pr view "$PR" --repo "$REPO" --json commits \
| jq -r '
"Total commits=" + ((.commits|length|tostring)),
([
.commits[]?
| {oid:(.oid//""|tostring), short:(.oid[0:7]//""), hasDco:(((.messageBody//"")|test("(?m)^Signed-off-by:\\s+.+<.+>$")))}
| select(.hasDco==false)
| .oid[0:7]
] | if length==0 then "DCO: NONE_MISSING" else ("Commits missing DCO Signed-off-by: " + (join(", "))) end),
# Show the commit messageBody for the missing ones (truncated)
([
.commits[]?
| select(((.messageBody//"")|test("(?m)^Signed-off-by:\\s+.+<.+>$"))|not)
| {oid:(.oid|tostring), messageBody:(.messageBody//""|gsub("\n";"\\n")|.[:600])}
] | .[])
'Repository: NVIDIA/cuopt
Length of output: 249
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
# Pull richer check data (often `statusCheckRollup` lacks context/state)
gh pr view "$PR" --repo "$REPO" --json checks,checkSuites,commits \
| jq -r '
"checks_count=" + ((.checks|length|tostring)),
(.checks[]?
| {name:(.name//null), conclusion:(.conclusion//null), status:(.status//null), detailsUrl:(.detailsUrl//null), checkSuite:(.checkSuite?.name//null), checkRun:(.checkRunId//null)}
| tojson
),
"",
"checkSuites_count=" + ((.checkSuites|length|tostring)),
(.checkSuites[]?
| {name:(.name//null), state:(.state//null), conclusion:(.conclusion//null), status:(.status//null), detailsUrl:(.detailsUrl//null)}
| tojson
)
'Repository: NVIDIA/cuopt
Length of output: 724
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
# Search comments timeline for /nvskills-ci and for any response text mentioning success/failure
gh pr view "$PR" --repo "$REPO" --json comments \
| jq -r '
([.comments[]?
| select((.body//""|test("/nvskills-ci\\b")))
| {login:.author.login, assoc:(.authorAssociation//null), body:(.body//""|gsub("\n";"\\n")|.[:200])}
])
'Repository: NVIDIA/cuopt
Length of output: 154
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
# Get PR head SHA
SHA="$(gh pr view "$PR" --repo "$REPO" --json headRefOid -q .headRefOid)"
echo "headRefOid=$SHA"
# Find any NVSkills-related success/failure text in PR comments
gh pr view "$PR" --repo "$REPO" --json comments \
| jq -r '
[ .comments[]?
| {login:.author.login, assoc:(.authorAssociation//""), body:(.body//"")}
| select((.body|ascii_downcase|test("nvskills")) or (.body|ascii_downcase|test("skills ci")) or (.body|ascii_downcase|test("validated")) or (.body|ascii_downcase|test("\\bpassed\\b|\\bpass\\b")) or (.body|ascii_downcase|test("\\bfailed\\b|\\bfail\\b")) or (.body|ascii_downcase|test("success")) or (.body|ascii_downcase|test("failure")) or (.body|ascii_downcase|test("validation")) or (.body|ascii_downcase|test("^/nvskills-ci\\b"))
]
| map({login,assoc,body:(.body|gsub("\n";"\\n")|.[:220])})
| sort_by(.login)
| .[]
'Repository: NVIDIA/cuopt
Length of output: 281
🏁 Script executed:
#!/bin/bash
set -euo pipefail
REPO="NVIDIA/cuopt"
PR="1392"
SHA="$(gh pr view "$PR" --repo "$REPO" --json headRefOid -q .headRefOid)"
echo "headRefOid=$SHA"
# List check-runs on the PR head commit; filter for NVSkills/skills-related names.
gh api "repos/${REPO}/commits/${SHA}/check-runs" --paginate \
| jq -r '
.check_runs? // []
| map({
name:(.name//null),
status:(.status//null),
conclusion:(.conclusion//null)
})
| map(select((.name|ascii_downcase|test("nvskills|skills"))))
| (if length==0 then [{name:"NONE_MATCHING",status:null,conclusion:null}] else . end)
| .[]
| tojson
'Repository: NVIDIA/cuopt
Length of output: 165
Fix required: missing DCO sign-off and NVSkills CI validation not evidenced for skills/** change
/nvskills-citrigger present (rgsl888prabhu [COLLABORATOR]), but no NVSkills/skills-related check-runs were found on the PR head SHA, so successful validation isn’t evidenced.- DCO sign-off missing: commit
f0ad3b7(noSigned-off-by:line in its message body).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@skills/cuopt-multi-objective-exploration/skill-card.md` at line 1, The PR
lacks a DCO sign-off on commit f0ad3b7 and shows no NVSkills CI validation for
changes under skills/**; fix by amending the commit message for f0ad3b7 to add a
"Signed-off-by: Your Name <you@domain>" line (git commit --amend --no-edit then
git push --force) so the DCO check can pass, and ensure the NVSkills CI runs
against the PR head SHA by re-triggering the /nvskills-ci workflow (or creating
a new empty commit/force-push) so the NVSkills/skills-related check-runs appear
on the PR; verify the check-runs complete successfully before marking resolved.
Comment on lines
+60
to
+69
| ## Evaluation Results: <br> | ||
| | Dimension | Num | `claude-code` | `codex` | | ||
| |---|---:|---:|---:| | ||
| | Security | 6 | 100% (+0%) | 100% (+0%) | | ||
| | Correctness | 6 | 82% (+27%) | 78% (+15%) | | ||
| | Discoverability | 6 | 67% (+33%) | 64% (+23%) | | ||
| | Effectiveness | 6 | 87% (+16%) | 80% (+8%) | | ||
| | Efficiency | 6 | 71% (+22%) | 63% (+13%) | | ||
|
|
||
| ## Testing Completed: <br> |
There was a problem hiding this comment.
Add blank lines around the table to satisfy markdownlint (MD058).
Suggested fix
## Evaluation Results: <br>
+
| Dimension | Num | `claude-code` | `codex` |
|---|---:|---:|---:|
| Security | 6 | 100% (+0%) | 100% (+0%) |
| Correctness | 6 | 82% (+27%) | 78% (+15%) |
| Discoverability | 6 | 67% (+33%) | 64% (+23%) |
| Effectiveness | 6 | 87% (+16%) | 80% (+8%) |
| Efficiency | 6 | 71% (+22%) | 63% (+13%) |
+
## Testing Completed: <br>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| ## Evaluation Results: <br> | |
| | Dimension | Num | `claude-code` | `codex` | | |
| |---|---:|---:|---:| | |
| | Security | 6 | 100% (+0%) | 100% (+0%) | | |
| | Correctness | 6 | 82% (+27%) | 78% (+15%) | | |
| | Discoverability | 6 | 67% (+33%) | 64% (+23%) | | |
| | Effectiveness | 6 | 87% (+16%) | 80% (+8%) | | |
| | Efficiency | 6 | 71% (+22%) | 63% (+13%) | | |
| ## Testing Completed: <br> | |
| ## Evaluation Results: <br> | |
| | Dimension | Num | `claude-code` | `codex` | | |
| |---|---:|---:|---:| | |
| | Security | 6 | 100% (+0%) | 100% (+0%) | | |
| | Correctness | 6 | 82% (+27%) | 78% (+15%) | | |
| | Discoverability | 6 | 67% (+33%) | 64% (+23%) | | |
| | Effectiveness | 6 | 87% (+16%) | 80% (+8%) | | |
| | Efficiency | 6 | 71% (+22%) | 63% (+13%) | | |
| ## Testing Completed: <br> |
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 61-61: Tables should be surrounded by blank lines
(MD058, blanks-around-tables)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@skills/cuopt-multi-objective-exploration/skill-card.md` around lines 60 - 69,
The markdown table under the "## Evaluation Results:" header violates MD058 by
not having blank lines surrounding it; fix by inserting a blank line immediately
after the "## Evaluation Results:" line and another blank line immediately
before the "## Testing Completed:" header so the table (the rows starting with
"| Dimension | Num | `claude-code` | `codex` |" through the final efficiency
row) is separated by blank lines from surrounding content.
| NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br> | ||
|
|
||
| (For Release on NVIDIA Platforms Only) <br> | ||
| Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br> |
There was a problem hiding this comment.
Use descriptive link text instead of “here” (MD059).
Suggested fix
-Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br>
+Please report quality, risk, security vulnerabilities, or NVIDIA AI concerns through the [NVIDIA Vulnerability Disclosure Program portal](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br> | |
| Please report quality, risk, security vulnerabilities, or NVIDIA AI concerns through the [NVIDIA Vulnerability Disclosure Program portal](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). <br> |
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 81-81: Link text should be descriptive
(MD059, descriptive-link-text)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@skills/cuopt-multi-objective-exploration/skill-card.md` at line 81, Replace
the ambiguous link text "here" with descriptive link text that explains the
destination and purpose (e.g., "report NVIDIA AI concerns on Intigriti" or
"report vulnerabilities on Intigriti"), updating the markdown in skill-card.md
where the current sentence contains "Please report quality, risk, security
vulnerabilities or NVIDIA AI Concerns
[here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail)". Ensure the
link text is meaningful for screen readers and satisfies MD059 by describing the
link target and context.
Iroy30
reviewed
Jun 4, 2026
Member
Iroy30
left a comment
Iroy30
left a comment
There was a problem hiding this comment.
Is the name appropriate? Cuopt might have multi objective capability later, maybe this might make agents confused?
Collaborator
Author
Iroy30
approved these changes
Jun 4, 2026
Member
Iroy30
left a comment
Iroy30
left a comment
There was a problem hiding this comment.
HAd a comment regarding naming but we could tackle that later when we are on the cusp of multi-objective support.
Collaborator
Author
|
/merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
skills/cuopt-multi-objective-exploration/SKILL.mdto mark the file as changed and trigger CI eval.