Skip to content

Commit 44f012a

Browse files
fix: make the build.mcpp host helper self-contained on musl AND MinGW (2026.7.28.2)
The compiled build.mcpp helper is exec'd by the host OS, outside anything mcpp controls — no wrapper, no injected LD_LIBRARY_PATH, no PATH guarantee. Whether it is actually runnable is a per-platform mechanism, and only two of the four were closed: Linux + glibc payload ok — host_base_flags bakes absolute payload paths into -Wl,-rpath Linux + musl (#295) bad — rpath cannot reach PT_INTERP, an absolute /lib/ld-musl-<arch>.so.1 no payload installs and no glibc distro ships -> execve ENOENT, surfacing as a bare exit 127 Windows PE (#299) bad — PE has no rpath at all, so the helper resolves libstdc++-6 / libgcc_s / libwinpthread via the process PATH -> STATUS_DLL_NOT_FOUND macOS ok — the system libc++ is always present Both gaps are the same defect (build_program.cppm re-derives the helper link policy instead of reusing the main build's, cf. prepare.cppm's musl -> linkage="static" default), so they get ONE decision point rather than two parallel predicates: a single staticHostHelper in run_build_program, with the chosen policy folded into the build-program cache identity so pre-fix helpers are rebuilt instead of reused. `-static` is appended to the final link argv only — never to `base`, which also feeds the bundled module's compile and precompile commands. Also: - process.cppm: keep the posix_spawnp error code and message in the captured output instead of collapsing to an unexplained exit 127 (both capture_exec and capture_exec_deadline). All three consumers of RunResult::output gate on exit_code first, so nothing downstream changes. - host_base_flags: stop emitting -Wl,-rpath on PE, where it is inert. - tests/e2e/168 (was 167 — taken on main by 167_build_defines_module_scan): discovers the installed musl-gcc payload instead of hardcoding 15.1.0, so it runs on the aarch64 host where the regression actually lives; asserts static + no PT_INTERP. Verified by negative control: reverting the -static push reproduces the exact #295 failure. - tests/e2e/97: hold the build.mcpp helper to the same standalone bar as the produced exe (import table + run with the toolchain bin off PATH). This runs under the real MinGW GCC 16.1.0 on Windows CI, which is #299's configuration. - ci-aarch64-fresh-install: append a PR regression gate that builds this source and runs e2e 168 against it. The checkout must come AFTER the fresh-install steps: .xlings.json declares an `mcpp` workspace pin, so a checkout present in $GITHUB_WORKSPACE makes `xlings install mcpp` workspace-scoped — which both voids the fresh-install charter (validating the pinned version, not the published one) and leaves bare `mcpp` unresolvable outside the workspace. Dropped the duplicated inline assertions, including `! readelf … | grep -q …`, which bash exempts from errexit and which could therefore never fail. Closes #295 Closes #299 Supersedes #300 Co-authored-by: Zhe Feng <zhefeng7110@outlook.com>
1 parent d0342db commit 44f012a

7 files changed

Lines changed: 224 additions & 117 deletions

File tree

.github/workflows/ci-aarch64-fresh-install.yml

Lines changed: 39 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ on:
2121
paths:
2222
- 'src/build/build_program.cppm'
2323
- 'src/platform/process.cppm'
24-
- 'tests/e2e/167_build_mcpp_musl_host_static.sh'
24+
- 'tests/e2e/168_build_mcpp_musl_host_static.sh'
2525
- '.github/workflows/ci-aarch64-fresh-install.yml'
2626
push:
2727
branches: [ main ]
@@ -44,10 +44,8 @@ jobs:
4444
# Verbose every mcpp invocation — cold bootstrap path (src/cli.cppm).
4545
MCPP_VERBOSE: "1"
4646
steps:
47-
- uses: actions/checkout@v4
48-
with:
49-
persist-credentials: false
50-
47+
# NB: the checkout deliberately comes LAST, after every fresh-install
48+
# step below — see the comment above it.
5149
- name: System info
5250
run: |
5351
uname -a
@@ -84,40 +82,16 @@ jobs:
8482
git clone --depth 1 https://github.com/openxlings/xim-pkgindex "$idx"
8583
grep -n "skipping relocation\|os.isfile(path.join(bindir" "$idx/pkgs/m/musl-gcc.lua" | head -2 || true
8684
87-
- name: Build current mcpp source for native regression tests
88-
run: |
89-
mcpp build --target aarch64-linux-musl
90-
self=$(find target/aarch64-linux-musl -type f -name mcpp | head -1)
91-
test -x "$self"
92-
self=$(realpath "$self")
93-
"$self" --version
94-
echo "MCPP_SELF=$self" >> "$GITHUB_ENV"
95-
9685
- name: Native build + run an `import std` program
9786
run: |
9887
work=$(mktemp -d); cd "$work"
99-
"$MCPP_SELF" new hello
88+
mcpp new hello
10089
cd hello
101-
cat > build.mcpp <<'EOF'
102-
#include <cstdio>
103-
int main() {
104-
FILE* marker = std::fopen("helper-ran", "w");
105-
if (!marker) return 2;
106-
std::fputs("ok\n", marker);
107-
std::fclose(marker);
108-
return 0;
109-
}
110-
EOF
11190
# default src uses import std (C++23)
112-
"$MCPP_SELF" build
113-
test -f helper-ran
114-
helper=target/.build-mcpp/build.mcpp.bin
115-
file "$helper"
116-
file "$helper" | grep -q 'statically linked'
117-
! readelf -lW "$helper" | grep -q 'Requesting program interpreter'
118-
out=$("$MCPP_SELF" run 2>/dev/null)
91+
mcpp build
92+
out=$(mcpp run 2>/dev/null || true)
11993
echo "program output: $out"
120-
bin=$(find target -type f -name hello | head -1)
94+
bin=$(find target -type f -path '*/bin/hello' | head -1)
12195
file "$bin"
12296
file "$bin" | grep -q "ARM aarch64" || { echo "expected aarch64 ELF"; exit 1; }
12397
@@ -127,14 +101,41 @@ jobs:
127101
# musl-static target is the published path, so build with --target.
128102
git clone --depth 1 https://github.com/mcpp-community/mcpp /tmp/mcpp-src
129103
cd /tmp/mcpp-src
130-
"$MCPP_SELF" self config --mirror GLOBAL 2>/dev/null || true
131-
"$MCPP_SELF" build --target aarch64-linux-musl
132-
m=$(find target/aarch64-linux-musl -type f -name mcpp | head -1)
104+
mcpp self config --mirror GLOBAL 2>/dev/null || true
105+
mcpp build --target aarch64-linux-musl
106+
m=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1)
133107
file "$m" | grep -q "ARM aarch64" || { echo "expected aarch64 mcpp"; exit 1; }
134108
"$m" --version
135109
git clone --depth 1 https://github.com/openxlings/xlings /tmp/xlings-src
136110
cd /tmp/xlings-src
137-
"$MCPP_SELF" build --target aarch64-linux-musl
138-
x=$(find target/aarch64-linux-musl -type f -name xlings | head -1)
111+
mcpp build --target aarch64-linux-musl
112+
x=$(find target/aarch64-linux-musl -type f -path '*/bin/xlings' | head -1)
139113
file "$x" | grep -q "ARM aarch64" || { echo "expected aarch64 xlings"; exit 1; }
140114
"$x" --version
115+
116+
# ── PR regression gate ────────────────────────────────────────────────
117+
# Everything above is the fresh-install charter: it must run exactly as a
118+
# new user's machine does. Check out only NOW — this repo's .xlings.json
119+
# declares an `mcpp` WORKSPACE pin, so with the checkout present in
120+
# $GITHUB_WORKSPACE `xlings install mcpp` installs workspace-scoped
121+
# instead of globally: the steps above would silently validate the pinned
122+
# version rather than the freshly published one, and bare `mcpp` stops
123+
# resolving anywhere outside the workspace.
124+
- uses: actions/checkout@v4
125+
with:
126+
persist-credentials: false
127+
128+
# The PR's own source, then the musl host-helper regression against it.
129+
# This is the only runner where a musl toolchain is the NATIVE one, so it
130+
# is the only place #295 can actually be reproduced.
131+
- name: Build current mcpp source for native regression tests
132+
run: |
133+
mcpp build --target aarch64-linux-musl
134+
self=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1)
135+
test -x "$self"
136+
self=$(realpath "$self")
137+
"$self" --version
138+
echo "MCPP_SELF=$self" >> "$GITHUB_ENV"
139+
140+
- name: "Regression: build.mcpp host helper is self-contained (#295)"
141+
run: MCPP="$MCPP_SELF" bash tests/e2e/168_build_mcpp_musl_host_static.sh

mcpp.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "mcpp"
3-
version = "2026.7.28.1"
3+
version = "2026.7.28.2"
44
description = "Modern C++ build & package management tool"
55
license = "Apache-2.0"
66
authors = ["mcpp-community"]

src/build/build_program.cppm

Lines changed: 43 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ import mcpp.platform;
1818
import mcpp.platform.process;
1919
import mcpp.toolchain.fingerprint; // hash_file / hash_string (FNV-1a, 16 hex)
2020
import mcpp.toolchain.linkmodel; // shared C-library / clang-cfg-bypass model
21-
import mcpp.toolchain.model; // Toolchain, PayloadPaths, is_clang/is_musl_target
21+
import mcpp.toolchain.model; // Toolchain, PayloadPaths, is_clang/is_musl_target/is_mingw_target
2222
import mcpp.toolchain.registry; // archive_tool
2323
import mcpp.toolchain.triple; // host_triple (MCPP_HOST contract value)
2424
import mcpp.ui;
@@ -160,8 +160,10 @@ std::string env_value(const std::string& name) {
160160

161161
// The host subset of flags.cppm's sysroot/runtime handling — enough to compile +
162162
// link a freestanding host program on a fresh sandbox (where bare `g++ file -o x`
163-
// can't find crt/libc). build.mcpp is host-only (skipped under cross), so we need
164-
// only the native cases; these are passed as separate argv tokens (no shell).
163+
// can't find crt/libc). `tc` is always a HOST-targeting toolchain: under a cross
164+
// --target, prepare.cppm resolves the spec a second time without the target axis
165+
// (host_tc_for_build_program) and passes that here, so the native cases are the
166+
// only ones needed. Passed as separate argv tokens (no shell).
165167
std::vector<std::string> host_base_flags(const mcpp::toolchain::Toolchain& tc) {
166168
std::vector<std::string> f;
167169
const auto lm = mcpp::toolchain::resolve_link_model(tc);
@@ -228,9 +230,14 @@ std::vector<std::string> host_base_flags(const mcpp::toolchain::Toolchain& tc) {
228230
if (!ar.empty()) f.push_back("-B" + ar.parent_path().string());
229231
}
230232
// Runtime lib dirs so the produced program can load private libs in-tree.
233+
// -L is link-time and wanted everywhere; rpath is an ELF-only concept —
234+
// this is the one host_base_flags branch a PE target reaches, where the
235+
// flag is inert and the self-containment answer is the static link in
236+
// run_build_program instead (#299).
231237
for (auto& d : tc.linkRuntimeDirs) {
232238
f.push_back("-L" + d.string());
233-
f.push_back("-Wl,-rpath," + d.string());
239+
if constexpr (mcpp::platform::supports_rpath)
240+
f.push_back("-Wl,-rpath," + d.string());
234241
}
235242
return f;
236243
}
@@ -585,12 +592,35 @@ std::expected<void, std::string> run_build_program(
585592
auto childEnv = contract_env(root, outDir, env);
586593
std::string ctxHash = contract_hash(childEnv);
587594

588-
const bool staticHostHelper = mcpp::platform::supports_full_static
589-
&& mcpp::toolchain::is_musl_target(tc);
595+
// ── Helper self-containment (the single decision point) ─────────────────
596+
// The compiled helper is exec'd by the host OS, outside anything mcpp
597+
// controls — no wrapper, no injected LD_LIBRARY_PATH, no PATH guarantee.
598+
// Making it runnable is a per-platform mechanism, and only two of the four
599+
// need a static link:
600+
// Linux + glibc payload — host_base_flags bakes absolute payload paths
601+
// into -Wl,-rpath; that already closes it, so leave it dynamic.
602+
// Linux + musl (#295) — rpath cannot reach PT_INTERP, an absolute
603+
// /lib/ld-musl-<arch>.so.1 that no payload installs and glibc distros
604+
// do not ship. Only a static link removes the interpreter entirely.
605+
// Windows PE (#299) — PE has no rpath, so a dynamic helper resolves
606+
// libstdc++-6 / libgcc_s / libwinpthread through the process PATH and
607+
// dies with STATUS_DLL_NOT_FOUND unless the user put the toolchain bin
608+
// there by hand. A static link is the only PATH-independent answer.
609+
// macOS — the system libc++ is always present.
610+
// Keep this the ONLY place that decides; the flag is appended to the final
611+
// link argv further down, never to `base`.
612+
const bool muslStaticHelper = mcpp::platform::supports_full_static
613+
&& mcpp::toolchain::is_musl_target(tc);
614+
const bool mingwStaticHelper = mcpp::toolchain::is_mingw_target(tc);
615+
const bool staticHostHelper = muslStaticHelper || mingwStaticHelper;
616+
617+
// Fold the policy into the compiler identity: a helper produced under an
618+
// older link policy must be rebuilt, not reused from the cache.
590619
std::string compilerIdentity = hostCompiler.string();
591-
compilerIdentity += staticHostHelper
592-
? "\nbuild-program-link=musl-static-v1"
593-
: "\nbuild-program-link=default-v1";
620+
compilerIdentity += "\nbuild-program-link=";
621+
compilerIdentity += muslStaticHelper ? "musl-static-v1"
622+
: mingwStaticHelper ? "mingw-static-v1"
623+
: "default-v1";
594624
std::string programHash = mcpp::toolchain::hash_file(src);
595625
std::string compilerHash = mcpp::toolchain::hash_string(compilerIdentity);
596626

@@ -647,10 +677,10 @@ std::expected<void, std::string> run_build_program(
647677
compileArgv.push_back("-x"); compileArgv.push_back("none");
648678
compileArgv.push_back((bdir / "mcpp.o").string());
649679
}
650-
// A dynamic musl helper requires /lib/ld-musl-<arch>.so.1 on the host,
651-
// which glibc distributions do not provide. Keep the host build program in
652-
// the musl toolchain's documented static world. This is link-only: `base`
653-
// also feeds the bundled module's compile/precompile commands above.
680+
// Self-contained helper link — see the staticHostHelper doctrine above.
681+
// Deliberately NOT in `base`: that also feeds the bundled module's
682+
// compile/precompile commands, where a link flag has no business (and for
683+
// Clang would perturb the default PIC/PIE codegen of mcpp.o).
654684
if (staticHostHelper) compileArgv.push_back("-static");
655685
compileArgv.push_back("-o"); compileArgv.push_back(bin.string());
656686
mcpp::ui::info("build.mcpp", "compiling");

src/toolchain/fingerprint.cppm

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ import mcpp.toolchain.detect;
1818

1919
export namespace mcpp::toolchain {
2020

21-
inline constexpr std::string_view MCPP_VERSION = "2026.7.28.1";
21+
inline constexpr std::string_view MCPP_VERSION = "2026.7.28.2";
2222

2323
struct FingerprintInputs {
2424
Toolchain toolchain;

tests/e2e/167_build_mcpp_musl_host_static.sh

Lines changed: 0 additions & 63 deletions
This file was deleted.
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
#!/usr/bin/env bash
2+
# requires: elf
3+
# 168_build_mcpp_musl_host_static.sh — a build.mcpp is a HOST executable that
4+
# the OS execs directly. Under a musl host toolchain it must be fully static:
5+
# a dynamic musl binary carries PT_INTERP=/lib/ld-musl-<arch>.so.1, an absolute
6+
# path no toolchain payload installs and no glibc distro ships, so execve fails
7+
# with ENOENT and the build dies as an unexplained exit 127 (#295).
8+
#
9+
# Runs standalone too — the aarch64 fresh-install workflow invokes it directly:
10+
# MCPP=/path/to/mcpp bash tests/e2e/168_build_mcpp_musl_host_static.sh
11+
set -euo pipefail
12+
13+
if [[ "$(uname -s)" != "Linux" ]]; then
14+
echo "SKIP: Linux ELF regression"
15+
echo "OK"
16+
exit 0
17+
fi
18+
19+
: "${MCPP:?MCPP must point to the mcpp binary under test}"
20+
21+
# Which musl toolchain to pin is a property of the machine, not of the test:
22+
# x86_64 dev boxes carry 15.1.0, the aarch64 runner resolves 16.1.0. Discover
23+
# the installed payload instead of hardcoding either — which is also why this
24+
# owns its own precondition (`requires: elf`, not `requires: musl`): run_all.sh's
25+
# musl capability probe is pinned to x86_64/15.1.0 and would false-skip on the
26+
# aarch64 host where this regression actually lives. Never auto-install here:
27+
# that would be a ~200 MB download per CI run (same rule as 28_target_static).
28+
find_musl_version() {
29+
local root gxx
30+
for root in "${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-musl-gcc" \
31+
"$HOME/.xlings/data/xpkgs/xim-x-musl-gcc"; do
32+
[[ -d "$root" ]] || continue
33+
gxx=$(ls "$root"/*/bin/*-linux-musl-g++ 2>/dev/null | head -1 || true)
34+
if [[ -n "$gxx" ]]; then
35+
# <root>/<version>/bin/<triple>-g++ → <version>
36+
basename "$(dirname "$(dirname "$gxx")")"
37+
return 0
38+
fi
39+
done
40+
return 1
41+
}
42+
43+
MUSL_VERSION=$(find_musl_version || true)
44+
if [[ -z "$MUSL_VERSION" ]]; then
45+
echo "SKIP: no musl-gcc payload installed — not testing the musl host helper"
46+
echo "OK"
47+
exit 0
48+
fi
49+
echo "musl toolchain: gcc@${MUSL_VERSION}-musl"
50+
51+
TMP=$(mktemp -d)
52+
trap 'rm -rf "$TMP"' EXIT
53+
cd "$TMP"
54+
"$MCPP" new app > new.log 2>&1 || {
55+
cat new.log
56+
echo "FAIL: mcpp new failed"
57+
exit 1
58+
}
59+
cd app
60+
61+
cat >> mcpp.toml <<EOF
62+
63+
[toolchain]
64+
default = "gcc@${MUSL_VERSION}-musl"
65+
EOF
66+
67+
cat > build.mcpp <<'EOF'
68+
#include <cstdio>
69+
int main() {
70+
FILE* marker = std::fopen("helper-ran", "w");
71+
if (!marker) return 2;
72+
std::fputs("ok\n", marker);
73+
std::fclose(marker);
74+
std::puts("mcpp:rerun-if-changed=build.mcpp");
75+
return 0;
76+
}
77+
EOF
78+
79+
"$MCPP" build > build.log 2>&1 || {
80+
cat build.log
81+
echo "FAIL: musl build.mcpp helper did not run on this host"
82+
exit 1
83+
}
84+
85+
helper=target/.build-mcpp/build.mcpp.bin
86+
test -f helper-ran || { cat build.log; echo "FAIL: build.mcpp did not run"; exit 1; }
87+
test -x "$helper" || { cat build.log; echo "FAIL: helper binary missing"; exit 1; }
88+
89+
file "$helper"
90+
file "$helper" | grep -q 'statically linked' || {
91+
echo "FAIL: musl host helper is not statically linked"
92+
exit 1
93+
}
94+
# The property that actually decides whether execve succeeds. NB: an `if`, not
95+
# `! readelf ... | grep -q ...` — bash exempts !-inverted pipelines from
96+
# errexit, so the negated form would never fail the test.
97+
if readelf -lW "$helper" | grep -q 'Requesting program interpreter'; then
98+
readelf -lW "$helper"
99+
echo "FAIL: musl host helper contains PT_INTERP"
100+
exit 1
101+
fi
102+
103+
echo "OK"

0 commit comments

Comments
 (0)