quality/coverage: enable Rust coverage in the LLVM pipeline - #772
quality/coverage: enable Rust coverage in the LLVM pipeline#772dcalavrezo-qorix wants to merge 2 commits into
Conversation
dcalavrezo-qorix
commented
Jul 24, 2026
There was a problem hiding this comment.
Pull request overview
This PR extends the unified llvm-cov coverage pipeline to include Rust crates (in addition to C++), by updating the Rust toolchain configuration, broadening the coverage scope to Rust sources/binaries, and enhancing the merger/reporter/justification tooling so Rust artifacts and paths are handled correctly.
Changes:
- Enable Rust coverage instrumentation under
bazel coveragevia Ferrocene/toolchain updates and Rust-specificcoverage.bazelrcflags. - Extend coverage scoping and collection to Rust binaries/crates (allowlist + baseline objects), and update merger/reporter to handle Rust-specific profdata/object discovery and path normalization.
- Update justification scanning to include
.rssources and avoid descending into Bazel output symlinks.
Reviewed changes
Copilot reviewed 5 out of 14 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| score/mw/com/impl/rust/com-api/com-api-runtime-lola/BUILD | Widens visibility to allow //quality/coverage to include this crate in coverage scoping. |
| score/mw/com/impl/rust/com-api/com-api-ffi-lola/BUILD | Widens visibility for coverage scoping access. |
| score/mw/com/impl/rust/com-api/com-api-concept/BUILD | Widens visibility for coverage scoping access. |
| score/mw/com/impl/plumbing/rust/BUILD | Widens visibility for coverage scoping access to plumbing Rust crates. |
| quality/coverage/llvm_cov/reporter.py | Adds Rust path normalization, rlib baseline expansion, and runfiles-based llvm-cxxfilt resolution. |
| quality/coverage/llvm_cov/merger.py | Resolves llvm-profdata for Rust runs and detects Rust test executables from the manifest. |
| quality/coverage/llvm_cov/BUILD | Adds llvm-cxxfilt as a data dependency for demangling support. |
| quality/coverage/justify.py | Scans .rs files for justification markers and skips Bazel convenience symlink trees. |
| quality/coverage/coverage.bazelrc | Adds Rust-specific rustc flags needed for usable coverage output (incl. continuous-mode profiling). |
| quality/coverage/coverage_scope.bzl | Extends scope collection to handle rust_binary via CrateInfo and removes debug prints. |
| quality/coverage/coverage_justifications.yaml | Adds explicit platforms fields to select justifications. |
| quality/coverage/BUILD | Explicitly adds Rust production crates to the overall coverage scope. |
| MODULE.bazel.lock | Updates module lock entries for the new Rust toolchain + Ferrocene coverage tools. |
| MODULE.bazel | Bumps score_toolchains_rust and updates Ferrocene coverage-tools URL/SHA to a newer release. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
castler
left a comment
There was a problem hiding this comment.
Great contribution. Thanks! Some comments from my side.
|
Thanks for addressing that. Do you maybe want to fixup your changes into the first commit and resolve the merge conflicts? Then we could try to merge that? |
@castler could you please re-review? changed the code according to your comments |
|
@dcalavrezo-qorix |
678c644 to
79e4e5c
Compare
Rebased onto main and squashed the review-fix commit into the first commit as requested. Notes:
|
Until now the unified llvm-cov pipeline reported C++ only: the Ferrocene toolchain registered for coverage runs did not declare llvm_cov, so rules_rust never instrumented Rust crates, no .rs file was in the scope allowlist, and justify.py did not scan Rust sources. Toolchain (score_toolchains_rust is already at 0.9.2 on main, whose ferrocene extension wires llvm-cov/llvm-profdata into the generated rust_toolchain): - bump ferrocene_toolchain_builder coverage-tools to 1.3.0, which ships llvm-cov/llvm-profdata/llvm-cxxfilt built from the same LLVM as rustc coverage.bazelrc (LLVM pipeline only; reset to empty for the gcov-based QNX pipeline via the list-typed extra_rustc_flags setting): - -Cllvm-args=-runtime-counter-relocation: continuous-mode profiling (LLVM_PROFILE_CONTINUOUS_MODE) otherwise writes no .profraw for Rust - -Clink-dead-code: keep unreferenced functions visible as uncovered - -Zcoverage-options=branch: branch coverage instrumentation (unstable flag; works because Ferrocene is a nightly-based rolling build) Pipeline: - coverage_scope.bzl: CrateInfo branch collects rust_binary sources and executables; rust_library was already handled via CcInfo; remove leftover debug prints - merger.py: resolve RUST_LLVM_PROFDATA (Bazel only sets LLVM_PROFDATA via the cc toolchain), terminating with a detailed error when no llvm-profdata is found; pick up the Rust test binary from the coverage manifest (ELF detection), skipping external/ toolchain binaries - reporter.py: normalize /proc/self/cwd/ covmap paths so the allowlist subtraction works with mixed C++ (absolute) and Rust (relative) paths; expand Rust rlib baseline archives into their .o members (llvm-cov rejects rlibs due to the leading lib.rmeta member); demangle via llvm-cxxfilt from runfiles (Rust v0/legacy and C++ symbols), terminating with an error when the declared data dependency is missing - justify.py: scan .rs files for COV_JUSTIFIED markers; skip bazel-* convenience symlinks - quality/coverage/BUILD: add the public Rust API crate (//score/mw/com/rust:score_com) to coverage_scope; the scope aspect reaches all Rust production crates through its transitive deps Known limitations: - 'manual'-tagged Rust tests are skipped by wildcard coverage runs; their crates report as 0% baseline unless invoked explicitly - Files without executable code (lib.rs-style re-export modules) carry no coverage regions and do not appear in reports (same as C++ headers)
justify.py validates 'platforms' as mandatory (as documented in llvm_cov/README.md), but the three gateway-* entries were added without it. Because generate_coverage_html.sh tolerates justify failures, the justification and effective-coverage processing has been silently skipped since then. With the field added, the 8 justified lines resolve again.
79e4e5c to
951d332
Compare
|
Thanks for the update and the accompanying notes.
If you already know what is necessary to fix this, feel free to add a commit to this PR. If not, please create a ticket and we will try to pick it up as soon as we can. |