ci: publish fork-PR test reports via workflow_run#4643
Conversation
GitHub forces GITHUB_TOKEN to read-only for pull_request runs from forks, so dorny/test-reporter's Checks API call (POST .../check-runs) returned 403 "Resource not accessible by integration" and failed every fork PR, even when all tests passed. Secrets are likewise absent on fork PRs, so the Azure OIDC login failed in the build-windows job. Split reporting out of the build: - ci.yml now runs with a read-only token and only uploads the .trx files as test-results-<platform> artifacts. Azure login and package signing are gated to refs/heads/main (publish-only, matching Push to MyGet). checks:write is dropped — the build no longer touches the Checks API. - test-report.yml is triggered by workflow_run when CI completes. It runs in the base-repository context with checks:write, downloads the test-results-* artifacts, and publishes one inline check per platform. This restores real test reports for fork PRs instead of swallowing them. Closes #4642 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
25ea816 to
aabb8f9
Compare
There was a problem hiding this comment.
Pull request overview
This PR hardens the repository’s CI for forked pull requests by moving test report publication into a separate workflow_run workflow (base-repo context with writable token) while keeping the PR CI workflow limited to building/testing and uploading .trx artifacts.
Changes:
- Replace inline
dorny/test-reportersteps inci.ymlwith.trxartifact uploads (test-results-Linux/test-results-Windows). - Add
test-report.ymltriggered viaworkflow_runto download those artifacts and publish per-platform check runs withchecks: write. - Restrict Azure login/signing to
mainonly (so fork PRs don’t fail due to missing secrets).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| .github/workflows/test-report.yml | New workflow_run workflow that publishes test reports from uploaded .trx artifacts using checks: write. |
| .github/workflows/ci.yml | Uploads test results as artifacts instead of publishing checks directly; removes checks: write from CI workflow permissions and ensures main-only publishing steps don’t run on PRs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # This workflow runs untrusted code from fork PRs, so it keeps a read-only token | ||
| # and never needs secrets. Test results are published by a separate workflow | ||
| # (test-report.yml, triggered via workflow_run) that runs in the base-repo | ||
| # context with checks:write. See issue #4642. |
There was a problem hiding this comment.
Good catch — reworded. The header comment now scopes the "no secrets / read-only" claim to PR (and especially fork) runs, and explicitly notes the main-only signing/publishing steps in build-windows are where secrets/OIDC are used. (commit 8c86567)
| permissions: | ||
| id-token: write | ||
| contents: read |
There was a problem hiding this comment.
Addressed by scoping id-token: write to the build-windows job only (the workflow default is now contents: read), so the Linux build job and any future job carry no OIDC. I kept signing/publishing in build-windows rather than splitting into a dedicated job because that path only runs on main and is untested-by-CI; restructuring it risks silently breaking releases for a marginal gain. Worth noting the fork escalation specifically isn't reachable: GitHub forces the token read-only for fork pull_request runs and will not honor id-token: write, so forks cannot mint an OIDC token here at all. The residual surface is trusted same-repo collaborators, and the Azure federated credential should be subject-constrained on the Azure side regardless. (commit 8c86567)
Per review feedback on #4643: - The workflow-level header comment claimed it 'never needs secrets', which is inaccurate for the main-only signing/publishing steps. Reword to scope the no-secrets/read-only claim to PR (and fork) runs. - id-token:write was granted workflow-wide, so it was active on the Linux build job and any future job. OIDC is only used by the main-only Azure login that code-signs packages, which lives in build-windows. Move the grant to that job so the default is contents:read and the Linux job carries no OIDC. (Fork PRs can't mint OIDC tokens regardless — GitHub forces the token read-only — so the residual surface is trusted same-repo actors.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes the fork-PR CI failures tracked in #4642 — and does it the right way, so fork PRs get real inline test reports instead of silently swallowed ones.
Problem
For
pull_requestruns from a fork, GitHub forcibly downgradesGITHUB_TOKENto read-only (untrusted code must not get write access to the base repo) and withholds secrets. Two steps inci.ymltherefore failed on every fork PR even when all tests passed:dorny/test-reportercalls the Checks API to publish results, which needschecks: write. With a read-only token it gets403 Resource not accessible by integrationand (defaultfail-on-error: true) fails the job.Azure Login via OIDCusessecrets.AZURE_*, which are empty on fork PRs, so login fails. This step only exists to sign/publish packages — amain-only path.Fix — split building from reporting (
workflow_runpattern)ci.yml(runs on the PR, read-only token, no secrets):Report Test Resultssteps withUpload Test Results— the.trxfiles are uploaded astest-results-Linux/test-results-Windowsartifacts.Azure Login via OIDCandSign packagestorefs/heads/main(matching the existingPush to MyGetguard).checks: writefrompermissions— the build no longer touches the Checks API.test-report.yml(new, triggered byworkflow_runwhen CI completes):checks: write, so it has a writable token even for fork PRs.test-results-*artifacts and publishes one inline check per platform (Test Results (Linux),Test Results (Windows)).fail-on-empty: falseso a build that fails before producing.trxdoesn't add a spurious red report.#4642 checklist
main(superseded here by the proper fix; Azure/signing gating carried over verbatim).workflow_runpattern for test reporting (the "optional, fuller fix").ci.ymlstep assumes secrets or a writable token on PRs —Azure Login,Sign packages, andPush to MyGetare allmain-only; the artifact uploads work fine with a read-only fork token.Note on rollout
workflow_runonly fires for the copy oftest-report.ymlon the default branch, so reports for fork PRs start working once this lands onmain. Until then, PR #4634 keeps itscontinue-on-errorstop-gap.🤖 Generated with Claude Code