Skip to content

Bound MSBuild compiler-process concurrency in the Windows CI build#6446

Closed
Fedr wants to merge 4 commits into
masterfrom
win-msbuild-process-cap
Closed

Bound MSBuild compiler-process concurrency in the Windows CI build#6446
Fedr wants to merge 4 commits into
masterfrom
win-msbuild-process-cap

Conversation

@Fedr

@Fedr Fedr commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Bound MSBuild's compiler-process concurrency in every CI step that invokes msbuild, by setting UseMultiToolTask=true and EnforceProcessCountAcrossBuilds=true as job-level environment variables (MSBuild picks environment variables up as global properties).

Plain msbuild -m + /MP gives no build-wide bound: every concurrently-building project independently fans its cl.exe processes out to the core count, so total compiler processes can reach projects-in-flight x cores, oversubscribing the runner's CPU and memory. With multi-tool-task scheduling and cross-build enforcement, all projects draw from one machine-wide pool limited to the core count (the MultiProcMaxCount default, deliberately not overridden here).

Covered call sites:

  • build-test-windows.yml — the solution build plus the MeshLibC2 / MeshLibC2Cuda C-bindings builds (job-level env)
  • pip-build.yml — the Windows wheel's solution build
  • update-win-version.yml — the example_plugin smoke builds

The env vars are inert everywhere else they're visible: CMake/Ninja legs schedule through a single job pool already, and dotnet build steps compile C#, not C++.

In MeshInspector CI, the same mechanism (there with an explicit process cap) made the Release MSBuild Build steps 23-26% faster on a 28-vCPU runner while also bounding peak commit charge; GitHub-hosted runners have the same oversubscription pattern at a smaller scale.

Test plan

  • full-ci: all Windows MSBuild legs (VS2019/2022/2026, Debug/Release) build green
  • Build step times on par with or better than master
  • pip-build.yml and update-win-version.yml are not exercised by PR CI; the change there is the identical env pair on the analogous solution builds

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fedr and others added 3 commits July 17, 2026 17:00
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…risons

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Fedr

Fedr commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Closing: the multi-tool-task setup does not bring faster Build times on the GitHub-hosted Windows runners. Same-CPU comparison (AMD EPYC 7763, Build step, full matrix): with EnforceProcessCountAcrossBuilds=true the MSBuild legs are ~3-5% slower than master; with enforcement off the times match master within noise, but then there is no build-wide process bound, so the configuration buys nothing. MTT also bypasses the /external:W0 suppression of vcpkg-header warnings from #6383 (ExternalIncludePath travels via the per-task EXTERNAL_INCLUDE environment variable, which the multi-tool task does not propagate to the cl.exe processes it spawns), which surfaced a C4389 in gtest.h on the VS2019 legs.

The mechanism stays in MeshInspector CI, where a high-vCPU runner measurably profits from the build-wide cap.

The unrelated fix this PR uncovered (unsigned/signed EXPECT_EQ comparisons in MRMeshBuilderTests.cpp) is pushed to master directly as 7dd8afd.

@Fedr Fedr closed this Jul 17, 2026
@Fedr
Fedr deleted the win-msbuild-process-cap branch July 17, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant