Add update-androidsdk-packages skill and skill-runner workflow - #12374
Add update-androidsdk-packages skill and skill-runner workflow#12374jonathanpeppers wants to merge 10 commits into
Conversation
…nner workflow Add a new project-local Copilot skill, update-androidsdk-packages, that automates refreshing the stable Android SDK package pins (versions, revisions, URLs, SHA-256 hashes) that src/androidsdk/androidsdk.csproj consumes from Configuration.props and src/androidsdk/androidsdk.targets, modeled on the update pattern in PR #12371. The skill enforces two hard rules: 1. Never touch the Android NDK (_XAAndroidNdk*/XAAndroidNdkHash*) -- it has its own release cadence and is out of scope here. 2. Never add a new Android platform API level to _PlatformPackage -- only refresh the revision/archive/hash of API levels already in the catalog. If Google has published a newer stable platform level than the highest one already present (e.g. platform 37.1 while the catalog tops out at 37.0), do not add it, but always surface that fact in the final summary every run, regardless of whether the request otherwise mentioned platform levels. Bundled resources: - scripts/fetch_repo_package.cs, scripts/sha256_of_url.cs -- C# file-based `dotnet run` apps (matching the ci-status skill's ci_failures.cs convention) that query Google's repository2-3.xml SDK manifest and compute authoritative SHA-256 hashes by downloading archives into a scratch temp file, since Google's manifests only publish SHA-1. Preview/alpha/beta/RC/canary releases are filtered with a word-boundary regex plus a <revision><preview> check, since channelRef alone is not a reliable stable/preview signal. - references/package-catalog.md -- mapping from each package family to its manifest path and Configuration.props/androidsdk.targets properties. - evals/evals.json -- realistic prompts covering normal updates and both hard-rule exclusions. Both scripts were reviewed via a rubber-duck pass that found and fixed: false-positive preview matches inside substrings like "sources", a missing <revision><preview> preview signal, channelRef being read from the wrong XML location, relative archive URLs not resolved against the manifest base URI, a Uri constructor call outside its try block, and a cleanup File.Delete that could mask the real success/failure outcome. Validated by building build-tools/Xamarin.Android.Tools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks.csproj (required for androidsdk.csproj to evaluate) followed by src/androidsdk/androidsdk.targets evaluation, live-testing both C# scripts against Google's real manifest, and confirming a clean diff. Also add a new gh-aw agentic workflow, skill-runner, that runs repository Copilot skills unattended on a weekly schedule or on demand: - workflow_dispatch exposes a `skill` dropdown (currently just update-androidsdk-packages) so a caller can pick which skill runs. When no skill is explicitly selected (scheduled runs, or dispatch left blank), a bootstrap step picks uniformly at random among the eligible skills -- the same pattern nightly-fix-finder uses for its scan scripts -- and records the choice for the prompt to read. - The prompt is skill-agnostic: it loads whichever SKILL.md was selected and follows it verbatim, consulting a "Known Skills" table only for workflow-level context (always-report rules, prerequisite bootstraps) that supplements rather than overrides the skill file. - Every run reports its outcome -- including pure no-ops, a newly published platform level found upstream, and any errors -- on a self-deduplicating tracking issue (close-older-issues, 30-day expiry). A PR is opened only when validated changes are produced, restricted to Configuration.props and src/androidsdk/androidsdk.targets. - An "Adding a New Skill" section documents the steps to wire in additional skills: dropdown + ELIGIBLE_SKILLS array entry, Known Skills table row, optional prerequisite-bootstrap elif branch, allowed-files update, and recompiling with `gh aw compile`. Compiled cleanly with `gh aw compile --approve` (the PAT-pool secrets this workflow references are the same ones already approved for nightly-fix-finder). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…unner Address three issues found in a rubber-duck review of the previous squashed commit: 1. fetch_repo_package.cs: revision sort/display key ignored <revision><preview>, so preview builds with different preview numbers (e.g. 36.0.0-rc1 vs 36.0.0-rc2) collapsed to the same sort key as each other and as the stable release, making "sorted newest first" unreliable under --all. The key is now a 4-tuple including preview number, with stable releases (preview == 0) always sorting above previews of the same major.minor.micro -- verified live against Google's manifest that build-tools 37.0.0 now sorts above 37.0.0-rc2 above 37.0.0-rc1. 2. skill-runner.md: an explicit workflow_dispatch `skill` input was accepted as long as `.github/skills/<input>/SKILL.md` existed on disk, bypassing the documented ELIGIBLE_SKILLS allowlist entirely. The bootstrap step now validates the input is an exact member of ELIGIBLE_SKILLS before using it, and fails fast if that array is ever empty instead of dividing by zero in `RANDOM % COUNT`. 3. SKILL.md carved out a rare exception allowing package.xml.in to change alongside Configuration.props/androidsdk.targets, but the skill-runner workflow's create-pull-request.allowed-files never included it -- a validated run could hit that rare path and then be unable to open its PR. Removed the exception: this skill's scope is package pins only, and automated runs must stay within the two files the workflow is authorized to touch. Recompiled skill-runner.lock.yml with `gh aw compile --approve`. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds repository automation to keep Android SDK package pins (revisions/URLs/SHA-256) in sync with Google’s stable SDK manifest by introducing a dedicated Copilot skill and an unattended “skill runner” gh-aw workflow.
Changes:
- Introduces the
update-androidsdk-packagesskill with detailed, hard-rule-guided procedure and bundled helper scripts for manifest querying + SHA-256 computation. - Adds a scheduled/on-demand
skill-runnergh-aw workflow (plus generated lock file) to execute eligible skills, open PRs for validated changes, and report outcomes via a tracking issue.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/skill-runner.md | New gh-aw workflow definition to select/run an eligible skill and bootstrap prerequisites. |
| .github/workflows/skill-runner.lock.yml | Generated compiled workflow output for skill-runner.md. |
| .github/skills/update-androidsdk-packages/SKILL.md | Skill playbook defining scope, hard rules, workflow, validation, and reporting requirements. |
| .github/skills/update-androidsdk-packages/scripts/sha256_of_url.cs | Helper script to download an archive and print SHA-256 for pinning. |
| .github/skills/update-androidsdk-packages/scripts/fetch_repo_package.cs | Helper script to query Google’s manifest for package revisions/archives and flag preview-looking entries. |
| .github/skills/update-androidsdk-packages/references/package-catalog.md | Reference mapping of package families to manifest paths and repo pin locations. |
| .github/skills/update-androidsdk-packages/evals/evals.json | Skill evaluation prompts/expectations to validate behavior and hard-rule exclusions. |
Replace `sha256.Hash!` with an explicit null-check that throws InvalidOperationException, per repo C# guidance banning the null-forgiving operator. Verified the script still hashes correctly against a real archive (build-tools_r37_linux.zip). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
/review |
|
✅ Android PR Reviewer completed successfully!
|
There was a problem hiding this comment.
⚠️ Needs Changes
Findings: 0 errors · 1 warning · 0 suggestions
The skill/workflow separation, allowlisted output files, NDK/platform guardrails, and relative-manifest URL handling are well designed. The archive hashing path should verify Google's manifest-provided SHA-1 and size before treating the downloaded bytes as authoritative and emitting the SHA-256 pin.
CI is still in progress: four checks plus CLA have passed, three platform builds are running, and the aggregate check is queued; no failures are currently reported.
Generated by Android PR Reviewer for #12374 · gpt56 · 129.2 AIC · ⌖ 8.85 AIC · ⊞ 25.6K
Comment /review to run again
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
|
@dalexsoto review |
dalexsoto
left a comment
There was a problem hiding this comment.
Five workflow blockers remain:
-
Manual dispatch can run from an arbitrary branch; unpinned checkouts execute that branch’s build/skill content and can open output from it. Require
main, pin both checkouts, and set the PR base explicitly. -
The skill installs sources into
sources/android-37.0, while consumers resolvesources/android-37; distinguish archive version 37.0 from the integer destination. -
Platform search results sort only by package revision, not API/extension level, so older platforms can be reported as newest. Sort platform paths by API and extension.
-
Weekly runs do not detect an existing open update PR, so unchanged main can produce duplicate PRs. Add stable-branch/open-PR deduplication.
-
cancel-in-progress: truecan cancel an earlier run before its unconditional report (or during output processing). Serialize without cancellation or otherwise guarantee final reporting.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
|
Addressed all five blockers from review 4931001292 in 1a3f6a1: workflows now check out and target main with queued non-cancelling concurrency; source destinations use integer API directories; platform discovery sorts by API/extension; and existing open skill-runner PRs are detected before creating another. The workflow was recompiled and the platform helper was verified against the live manifest. |
dalexsoto
left a comment
There was a problem hiding this comment.
Two workflow blockers remain:
-
Manual dispatch still does not reject non-main refs. The selected ref controls the workflow/activation content, an activation checkout remains unpinned, and generated PR-context handling can replace the main checkout with branch content. Reject non-main dispatches before activation and disable PR-context checkout for this workflow.
-
Open-PR deduplication is only an agent prompt using a default-limited
gh pr listand an “equivalent” judgment. It can miss older PRs or allow multiple skill-runner updates. Use a deterministic paginated/search-filtered key and enforce it immediately before PR creation.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
|
Addressed both workflow blockers: manual dispatches are rejected unless the ref is |
dalexsoto
left a comment
There was a problem hiding this comment.
Three workflow blockers remain:
-
The non-main dispatch rejection runs only after
pre_activation, PAT-pool selection, and activation have already processed branch-controlled workflow content. Enforce main at the earliest job/environment conditions before activation. -
Generated
aw_contexthandling can still replace the pinned main checkout with a PR branch on manual dispatch. Reject PR ambient context or disable that generated checkout. -
PR deduplication remains prompt-only and bounded to
gh pr list --limit 100; it is neither exhaustive nor enforced in the trusted creation path. Add deterministic exhaustive uniqueness enforcement immediately before PR creation.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
|
Fixed the remaining blockers by rejecting non-main workflow_dispatch runs before activation, forbidding PR aw_context on manual dispatch, and enforcing the exact-match open-PR dedupe gate before any pull request creation. |
dalexsoto
left a comment
There was a problem hiding this comment.
Two workflow blockers remain:
-
The
mainguard is only onactivation;pre_activationandpat_poolstill run first without the non-mainworkflow_dispatchcondition, so branch-controlled compiled workflow content reaches the protected environment and PAT-selection path before rejection. Apply the main-ref gate to the earliest jobs (includingpre_activation/pat_pool), not only their dependent activation job. -
Open-PR deduplication is still agent-prompt logic and its prescribed command is bounded by
gh pr list --limit 100; “repeat/paginate” does not make that command exhaustive, and nothing in the trusted safe-output creation path enforces the exact workflow/skill key. Add deterministic exhaustive uniqueness enforcement immediately before PR creation outside the agent’s discretionary prompt execution.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
|
You were right that the previous change only restated the controls in the agent prompt and gated activation too late. Fixed in 354bd42: the ref/PR-context checks are now injected into pre_activation via on.steps, and an exact GitHub search filter prevents any run when an open [skill-runner] update PR for update-androidsdk-packages already exists. The compiled lock file was regenerated. |
dalexsoto
left a comment
There was a problem hiding this comment.
Four workflow blockers remain:
-
The non-main/PR-context checks are steps inside
pre_activation, so the job enters the protectedcopilot-pat-poolenvironment and runs setup, membership, and search actions before rejecting branch-controlled dispatches. Put these conditions on the job itself before environment allocation (skill-runner.lock.yml:1712-1776). -
PR deduplication is neither exact nor enforced at creation: the early title search omits
base:main, safe outputs only require the generic[skill-runner]prefix, and the trusted creation handler does not recheck the workflow/skill key. Enforce the exact key againstmainimmediately before creation (skill-runner.lock.yml:1752-1764, 1885-1903). -
A skip-query match disables activation, which skips the agent and
conclusion; the required no-op tracking report and platform-catalog status are never emitted. Route duplicate matches through the mandatory reporting path (skill-runner.lock.yml:1721-1764, 1127-1145). -
The skill requires both
androidsdk.csprojvalidations with--no-restore, but a fresh runner only restores the unrelated BootstrapTasks graph.androidsdk.csproj/openjdk.csprojtherefore have no assets file and validation cannot run. Restore that project first or remove--no-restorefrom the initial validation (SKILL.md:135-151).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 49de4c35-38d9-4c5a-a41b-b6b017260349
|
Addressed the four latest blockers in 43b5204 and 1e17348: added a custom workflow_guard job required before pre_activation so invalid dispatches never enter the PAT environment; replaced skip-if-match with a base=main, paginated exact-title check that emits a safe noop before agent/PR creation and preserves conclusion reporting; and added dotnet restore src/androidsdk/androidsdk.csproj before --no-restore validation. The compiled lock file was regenerated. |
Why
src/androidsdk/androidsdk.csprojpins Android SDK package versions/revisions/URLs/SHA-256 hashes inConfiguration.propsandsrc/androidsdk/androidsdk.targets. Google republishes tool revisions on its own cadence (see PR #12371 for the last manual refresh), and keeping these two files in sync is a repetitive, error-prone task: it requires querying Google's manifest, distinguishing stable from preview releases, and recomputing SHA-256 hashes per host/arch since Google only publishes SHA-1.What this adds
update-androidsdk-packagesCopilot skill (.github/skills/update-androidsdk-packages/)Automates the refresh, matching the pattern from PR #12371. It enforces two hard rules:
_XAAndroidNdk*/XAAndroidNdkHash*) - out of scope, separate release cadence._PlatformPackage- only refresh revisions/hashes of API levels already in the catalog. If a newer stable platform level exists upstream than what's in the catalog (e.g. platform 37.1 while the catalog tops out at 37.0), the skill must always surface that fact in its summary, even if nothing else changed.Bundled resources:
scripts/fetch_repo_package.cs,scripts/sha256_of_url.cs- C# file-baseddotnet runapps (matching theci-statusskill's convention) that query Google'srepository2-3.xmlmanifest and compute authoritative SHA-256 hashes by downloading archives to a scratch temp file. Preview/alpha/beta/RC/canary releases are filtered out using a word-boundary regex plus the<revision><preview>element, sincechannelRefalone isn't a reliable signal.references/package-catalog.md- maps each package family to its manifest path andConfiguration.props/androidsdk.targetsproperties.evals/evals.json- realistic prompts covering normal updates and both hard-rule exclusions.Validated by building
Xamarin.Android.Tools.BootstrapTasks.csproj(required forandroidsdk.csprojto evaluate), live-testing both scripts against Google's real manifest, and a rubber-duck review that found and fixed several bugs (false-positive preview matches inside substrings, relative archive URLs not resolved against the manifest base URI, preview revisions colliding in the sort key, and more).skill-runneragentic workflow (.github/workflows/skill-runner.md)A
gh-awworkflow, modeled onnightly-fix-finder, that runs repository Copilot skills unattended on a weekly schedule or on demand:workflow_dispatchexposes askilldropdown (currently justupdate-androidsdk-packages, validated against anELIGIBLE_SKILLSallowlist). When no skill is explicitly selected, one is picked at random among eligible skills.SKILL.mdwas selected and follows it verbatim.Configuration.propsandsrc/androidsdk/androidsdk.targets.Review notes
skill-runner.lock.ymlis generated viagh aw compile; don't hand-edit it.nightly-fix-finder.md.