diff --git a/.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md b/.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md new file mode 100644 index 00000000..92376ade --- /dev/null +++ b/.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md @@ -0,0 +1,597 @@ +# xlings 作为 mcpp 的运行时底座:运行时身份、链接契约与环境契约 + +**日期**:2026-08-07 +**性质**:设计提案,待 review 后再实施。本文不改任何代码。 +**触发**: +- mcpp#375(产物烙私有 glibc PT_INTERP ⇒ 不可分发;系统 loader 包装 ⇒ `/proc/self/exe` 失效) +- mcpp#352(GLFW/OpenGL 静默 exit 255:沙箱 glibc 2.39 加载不了宿主 Mesa) +- xlings 侧新能力已落地:glibc 2.44、22 包 hermetic 图形栈(`xim:graphics`)、subos 自描述(`subos_info`) + +**关联(xlings 仓)**: +- `.agents/docs/2026-08-05-ecosystem-three-tier-and-composable-distro.md` —— 三分层定位,**开放问题 K:mcpp 消费 subos 还是消费 platform 描述**。本文回答它。 +- `.agents/docs/2026-08-06-subos-architecture-proposal.md` —— 七条规则 R1–R7,本文全程引用 +- `.agents/docs/2026-08-05-userspace-distro-hermetic-strategy.md` —— 运行时边界 + +**关联(mcpp 仓)**: +- `.agents/docs/2026-08-02-issue336-pr142-analysis.md` —— `cxx_runtime` 契约层。**本文是它在 libc 轴上的同族补全。** +- `.agents/docs/2026-07-07-hermetic-toolchain-link-model-design.md` —— `linkmodel.cppm` 的由来 + +--- + +## 0. TL;DR + +**一句话根因:mcpp 把 xlings 当成「一套需要自己反推的目录约定」,而不是「一个可以查询的运行时」。** + +后果是可预测的:xlings 每前进一步,在 mcpp 侧都落成**一次改代码**,而不是**一次改数据**。glibc 2.44 如此,图形栈如此,下一个能力也会如此。 + +三条缝,按依赖顺序: + +| | 缝 | 现状 | 目标 | +|---|---|---|---| +| **S1** | **运行时身份** | 从目录布局反推「哪个 libc」,且**无版本** | 读 subos 的 `subos_info.runtime`(`glibc@2.39`),成为一等轴 | +| **S2** | **分发路径** | 三条 hermetic 路径都存在,但用户找不到;其中一条自己是坏的 | 修坏的那条 + 让三条可发现。**不加「链宿主 libc」的开关** | +| **S3** | **环境契约** | mcpp 产物拿不到 subos 的 env,图形栈靠 mcpp 代码硬扛 | 消费 `subos_info.envs`;图形栈以**依赖**而非代码到达 | + +**关键结论:P0 需要 xlings 零改动。** `subos_info` 已经存在,subos view 已经填充好(`subos/default/lib/ld-linux-x86-64.so.2` 实测在位)。唯一需要 xlings 配合的是**把 installer 已经算出来、当前只活在进程里的 `resolved_deps`/`deps_exports` 持久化**——那是 P1。 + +**多维评估见 §11**(实现代价 / 用户 / 稳定性 / 跨平台 / 简洁 / 兼容性,每项带实测数字)。三条要点: + +- **本文被推翻过一整节**(§3-S2):初稿要补一条 `c_runtime` 轴,其 `host-coupled` = 链宿主 libc。已实现、全绿、然后整条撤销。理由与教训写在那一节,**比结论更值得读** +- **零 BMI/对象缓存失效** —— fingerprint 是 compile-side,分发相关的改动都只碰链接与打包 +- **收益 ≈ 全在 Linux**;#352 的图形栈迁移是其中收益最大、代价最小的一条 + +--- + +## 1. 实测现状(不是推断) + +本节每一条都在本机跑过,命令附在后面,便于 review 时复核。 + +### 1.1 产物里烙进了四个独立的版本 pin + +``` +$ mcpp new hello && cd hello && mcpp build +$ file target/x86_64-linux-gnu/*/bin/hello +… interpreter /home/speak/.mcpp/registry/data/xpkgs/xim-x-glibc/2.39/lib64/ld-linux-x86-64.so.2 +$ readelf -d target/x86_64-linux-gnu/*/bin/hello | grep RUNPATH + RUNPATH [ …/xim-x-llvm/22.1.8/lib/x86_64-unknown-linux-gnu + : …/xim-x-llvm/22.1.8/lib + : …/xim-x-gcc-runtime/15.1.0/lib64 + : …/xim-x-glibc/2.39/lib64 ] +``` + +一个 hello world 产物同时钉死了:**home 绝对路径**、**glibc 2.39**、**llvm 22.1.8**、**gcc-runtime 15.1.0**。 + +这正是 xlings `libs/graphics.lua:47-54` 刚刚明文废弃的反模式: + +> `${pkgdir}` 是声明包自己的载荷,它**钉死一个版本目录**:升级 mesa 会让消费者记录的 env 指向旧的那个。subos 视图才是稳定的间接层——就是 `/run/opengl-driver` 在 NixOS 扮演的角色。 + +mcpp 烙的就是 `${pkgdir}` 等价物。 + +### 1.2 稳定间接层已经存在,而且已经填充好 + +``` +$ ls -l ~/.mcpp/registry/subos/default/lib/ +ld-linux-x86-64.so.2 -> …/xim-x-glibc/2.39/lib64/ld-linux-x86-64.so.2 +crt1.o crti.o crtn.o -> …/xim-x-glibc/2.39/lib64/… +libc.so.6 libc.so libm.so.6 … -> …/xim-x-glibc/2.39/lib64/… +libc++.so.1 libc++abi.so.1 -> …/xim-x-llvm/22.1.8/lib/… +libatomic.so.1 libasan.so.8 -> …/xim-x-gcc/16.1.0/lib64/… +``` + +loader、CRT、libc、C++ 运行时**全部在视图里**,按活动版本指向载荷。mcpp 一个都没用。 + +> ⚠️ 注意布局差异:载荷里 loader 在 `lib64/`,视图里在 `lib/`。mcpp 现有的 `{lib64, lib}` 顺序探测(`probe.cppm:346-349`、`linkmodel.cppm:249-258`)**碰巧两边都能命中**——因为它回退到 `lib`。但「碰巧能命中」正是本文反复要指出的那类脆弱:两个布局不同的树共用一份约定探测,靠的是回退顺序恰好合适。视图新增一个 `lib64` 就会翻。这条要在 P0 验证门里钉住(V5)。 + +### 1.3 mcpp 的 sandbox subos 不描述自己 + +``` +$ python3 -c "import json;d=json.load(open('$HOME/.mcpp/registry/subos/default/.xlings.json'));print(list(d.keys()), len(d['workspace']))" +['workspace'] 356 +``` + +356 个版本条目,**零运行时身份、零环境声明**。没有 `subos_info` 块。 + +而 xlings `src/core/subos/manifest.cppm:16-20` 的模块注释,点名了这件事: + +> 一个程序需要三样:bootstrap(PT_INTERP + CRT + libc)、discovery(PATH + RPATH)、configuration(env vars)。xlings 有前两样——glibc + elfpatch、xvm + shims——第三样什么都没有。**That is the gap behind mcpp-community/mcpp#352**:一个 GLFW 二进制链接得好好的,exit 255,因为没人告诉它 GL 驱动在哪。 + +xlings 已经把这个缺口补上了(`subos_info`,schema 1)。**mcpp 的 subos 没有拿到它。** + +### 1.4 xlings 的权威答案只活在进程里 + +`src/core/xim/installer.cppm:2394-2450` 已经为**每一个** runtime 依赖计算并记录了完整档案(遵守 R1「权威记录必须是全量的」),包括 `install_dir`、`libdirs`(`{lib64, lib}` 约定**只在这里**应用一次)、`loader`、`abi`。 + +但落到磁盘上的只有: + +``` +$ cat ~/.mcpp/registry/data/xpkgs/xim-x-glibc/2.39/.xpkg-install.json +{ "os": "linux", "version": "2.39", "xlings_version": "2026.8.2.1" } +``` + +三个字段。`exports` 一个都没落盘。 + +这解释了 mcpp `linkmodel.cppm:182-188` 当年为什么拒绝它: + +> 第三个来源——installer 写的持久化 `.xpkg-exports.json` 声明元数据——评估后移除了:它唯一的消费者只有这个 resolver,而上面两个来源已经覆盖了每一个真实载荷。 + +**那个判断在当时是对的,今天不再对**。理由见 §2.2。 + +### 1.5 图形栈:mcpp 侧仍是宿主借用 + +mcpp-index `pkgs/c/compat.glx-runtime.lua` 至今把宿主 `/usr/lib/x86_64-linux-gnu/libGL.so.1` 等 symlink 进 `mcpp_generated/glx_runtime/lib`,再经 `[runtime] library_dirs` 进 RUNPATH(`flags.cppm:698-706`)。 + +这就是 #352 的直接成因:宿主 Mesa 要 `GLIBC_2.43`,载荷 glibc 是 2.39。而 hermetic 策略已经把这个文件点名为**要淘汰的样本**。 + +同时,xlings 侧的替代品**已经可用**:`xlings install graphics` —— 22 包 hermetic 栈 + NVIDIA/WSL2 哨兵,一条命令五种宿主形态,无条件分支。 + +### 1.6 `/proc/self/exe` 陷阱不是用户的 workaround —— 是 mcpp 自己在生产它 + +这是本次调研里最出乎意料的一条,值得单独列。 + +#375 描述的第三条症状(用 `ld.so --library-path` 启动后 `/proc/self/exe` 指向 loader)读起来像是用户自己想出来的绕法。**但 mcpp 的 `self-contained` 打包模式生产的就是这个 wrapper**,`docs/02-pack-and-release.md:120-127` 原样写着: + +```sh +exec "$here/lib/ld-linux-x86-64.so.2" --library-path "$here/lib" "$here/bin/myapp" "$@" +``` + +文档给的理由是对的(ELF 规范禁止 `PT_INTERP` 用 `$ORIGIN`),但**后果没有写**:凡是用 `mcpp pack --mode self-contained` 分发的程序,`/proc/self/exe` 全部指向 loader,`/proc/self/cmdline` 全部混入 `--library-path`。所有「在 exe 旁边找资源」的逻辑静默失效——字体、assets、随包分发的辅助二进制。 + +而 mcpp 自己就依赖这个机制:`src/platform/fs.cppm:106` 读 `/proc/self/exe` 定位自身。 + +全仓 grep `proc/self/exe` 只有那一处实现,**文档里零处提及这个陷阱**。所以 #375 请求的两条文档补充里,(a)「私有 glibc 对分发的影响」其实已经写了(`02-pack-and-release.md:3-6` 开篇就是),(b)「系统 loader 包装后的 `/proc/self/exe` 陷阱」**确实没有,而且比提问者以为的更严重**——它不是一条使用建议,是一个在售模式的已知缺陷。 + +--- + +## 2. 根因 + +### 2.1 一个问题,mcpp 侧有 N 个回答者 + +「C 运行时在哪、loader 是哪个、哪些目录进 RUNPATH」这一个问题,mcpp 侧今天有六个独立推导: + +| # | 位置 | 怎么答的 | +|---|---|---| +| 1 | `probe.cppm:333-349` | `find_sibling_tool(compilerBin,"glibc")` 爬父目录 → `lib64` 不存在则 `lib` | +| 2 | `linkmodel.cppm:194-243` | 五行 arch→loader 文件名硬表,不中则 glob `ld-*.so*` | +| 3 | `linkmodel.cppm:215-221` | `distro_loader_path`:x86_64 → `/lib64/`,其余 → `/lib/` | +| 4 | `post_install.cppm:162-200` | 扫 clang cfg 文本找 `/ld-linux-` 反推烙定的 loader | +| 5 | `pack.cppm:650-653` | 从 `ldd` 输出取 loader soname,再拼 `/lib64/` 或 `/lib/` | +| 6 | `probe.cppm:190-223` | `lib` / `lib64` / `lib/` 三种约定拼 runtime 目录 | + +这**正是** xlings 侧 `2026-08-06-subos-architecture-proposal.md` §1 诊断出的 P1「一个问题有多个回答者」,只是发生在河的下游。而 xlings 已经从自己那边删到了一个(R2:约定只在写端应用,读端永远不猜)——mcpp 侧还有六个。 + +判据(R3):**修复如果是「增加一条路径」而不是「移除一条」,它是 workaround。** 下面的设计要能删掉其中的 4–5 条,否则不合格。 + +### 2.2 为什么「拒绝声明元数据」的判断需要翻转 + +`linkmodel.cppm` 当年的理由是「唯一的消费者只有这个 resolver」。今天的消费者清单是: + +| 消费者 | 需要什么 | 约定能答吗 | +|---|---|---| +| 链接模型 | loader、libdirs、CRT 目录 | 能(勉强) | +| **运行时身份**(新) | glibc **版本**、ABI family | **不能**——目录名不是契约 | +| **subos 视图寻址**(新) | 视图里的 loader 路径(在 `lib/`,载荷在 `lib64/`) | **靠回退顺序碰巧能**(§1.2)——即两个不同布局共用一份猜测 | +| **图形/env 契约**(新) | 哪些 env、哪些值 | **不能** | +| `mcpp pack` | 目标机 loader | 能(硬表) | +| `mcpp doctor` | 悬空链接、版本偏斜 | 部分 | + +从一个消费者变成六个,而新增的三个**约定原理上答不了**。R2 的判据在这里是决定性的:一句「没有就自己猜」等于授权每个读端各自实现一份猜测,而读端会随时间增加——mcpp 侧已经从 1 增加到 6。 + +### 2.3 缺的那一层:libc 轴的契约层在错误的生命周期上 + +这是最关键的一条,而且 mcpp 自己的代码已经把它写出来了。 + +`src/build/distribution.cppm:73-77`: + +> **NOTE ON SCOPE**:契约管的是 C++ 运行时(stdlib + 它的 ABI 与 unwinder)。**libc 轴是分开的,归 `linkage`/`--static` 管**,部署地板是第三条轴。 + +三层模型 `Role → Contract → Mechanism` 在 C++ 轴上是完整的: + +``` +Role Distributable | Test | Intermediate ← 内在,用户写不了 +Contract SelfContained | ToolchainCoupled | HostCoupled ← [build] cxx_runtime +Mechanism (contract × stdlib × 二进制格式) → 链接 flags ← total function +``` + +**libc 轴上只有 Mechanism 那一层,而且分散在两个生命周期**:构建期 `--static`,打包期 `--mode`。用户在构建期能说的只有机制,说不了意图(「这个产物要能在没装 mcpp 的机器上跑」)。 + +而 #336 的分析文档写清了这个形状为什么必然出错: + +> 那个 bool 拼的是**机制**(「静态链接 stdlib」),它膨胀成三种不同的平台含义——包括在 Linux/libc++ 上静默无操作:声称 static、产出 toolchain-coupled。契约拼的是**意图**。 + +**这个观察是对的,但从它推出的结论曾经是错的。** 初稿由此推出「那就补一条 `c_runtime` 轴」——而这条轴唯一的新能力是把产物链到宿主 libc,即策略上被禁的那一侧。**一个真实的架构不对称,不构成做一件被禁的事的理由。** 正确的读法见 §3-S2:用户要的是「能分发」,而 hermetic 的分发路径已经有三条。 + +这一段保留,是因为那个不对称本身仍然真实,只是它的正确用途是**解释为什么 `mcpp pack` 的默认模式是宿主耦合的**(§3-S2 末尾那笔记账),而不是给 build 期再开一个口子。 + +**准确说,不是「没有机制」——是机制在错误的生命周期上。** `mcpp pack` 有一个成熟的两轴模型(`docs/02-pack-and-release.md:8-37`:target(libc)× mode(bundling depth),四个模式 `system` / `vendored` / `self-contained` / `static`),`vendored` 默认就会把 PT_INTERP 重指到 `/lib64/ld-linux-*.so.2`(`pack.cppm:649-658`)。所以 **#375 的第 1、2 条症状今天有受支持的答案**。 + +真正的不对称是这个: + +| 轴 | 契约在哪一期可声明 | 怎么实现 | +|---|---|---| +| C++ 运行时 | **构建期**,`[build] cxx_runtime` | 链接时给对 flags | +| C 运行时 / loader | **只有打包期**,`mcpp pack --mode` | 链接后用 patchelf 改写 | + +同一类决策(「产物对运行它的机器承诺什么」),一条在构建期声明、一条只能在打包期补救。后果有三个,都是实测的: + +1. `mcpp build` 的产物看起来像交付物、其实不是,而**这件事只有读到 pack 文档才知道**——#375 的提问者没走到那一步,直接去套了 `ld.so` 包装 +2. `mcpp run` 跑的永远不是将要分发的那个配置,所以分发问题只能在 pack 之后才暴露 +3. patchelf 改写是**事后**的:它改得了 PT_INTERP 与 RUNPATH,改不了「链接时就该按目标契约选 RUNPATH 形状」这件事,于是 `self-contained` 只能退回 wrapper 脚本——**即 §1.6 那个缺陷的来源** + +> **推论(值得单独强调)**:`/proc/self/exe` 那条症状**不是私有 glibc 的后果,是 wrapper 的后果**;而 wrapper 是「契约无法在链接期表达」的后果。只要 PT_INTERP 指向一个在目标机上真实存在的路径,就不需要 wrapper,`/proc/self/exe` 自然正确。**修契约,第三条症状自己消失——不需要为它单独设计任何东西。** + +### 2.4 R6:mcpp 两个用途都绑了载荷,而只有一个该绑 + +xlings 侧 R6:**内部消费者绑定 payload,不绑定视图。** 三层的消费者不同: + +| 层 | 谁该消费 | mcpp 今天 | +|---|---|---| +| payload `data/xpkgs//` | xlings/libxpkg 自身;**构建期**的 flags | ✅ 用了(正确) | +| subos 视图 `subos//{bin,lib,usr}` | 用户,以及**用户运行的程序** | ❌ 没用 | +| 每个产物的 RPATH/INTERP | 动态加载器 | ❌ **烙的是 payload** | + +mcpp 在**构建期**绑载荷是对的(版本精确、fingerprint 稳定、R6 合规)。 +mcpp 在**运行期**绑载荷是错的——那一层的正确锚点是视图。 + +**一句话:mcpp 用一个地址服务了两个生命周期不同的用途。** 这是所有三个症状的公共上游。 + +--- + +## 3. 设计 + +### S1 — 运行时身份(RuntimeBinding) + +**mcpp 停止反推「哪个 libc」,改为读。** + +新增一条一等轴 `RuntimeBinding`,词汇与 xlings **逐字一致**(`glibc@2.39` / `musl@1.2.5` / `macos_sdk@14.0` / `ucrt@…`),而不是新造一套。 + +解析优先级(**每一级都必须显式,不允许「缺省即约定」**——A2): + +1. `--runtime ` CLI +2. `[target.].runtime` / `[build].runtime` +3. **活动 subos 的 `subos_info.runtime`** ← 新的权威源 +4. 载荷实测(仅当 subos 无 `subos_info`,即老 subos 的降级路径,且**必须打一行可见提示**) + +配套:`abi.cppm:71-106` 的 `libc` 维度从**无版本**(`"glibc"`)升级为**带版本**(`glibc@2.39`),`family_of` 的映射与 xlings `manifest.cppm:81-92` 对齐——但**只在一处推导**。 + +> **这条同时解掉一个隐性重复账**:mcpp `abi_profile()` 从 target triple 推 `*-linux-gnu → glibc`,xlings `family_of()` 从 runtime 推 `glibc@2.39 → linux-x86_64-glibc`。同一个概念两处推导,而 mcpp 那份没有版本——所以 mcpp 今天**表达不了**「这个依赖需要 glibc ≥ 2.39」。多 glibc 一到,它就是下一个 bug 源。 + +**删除**:`probe.cppm` 的 glibc 目录爬升与 `{lib64,lib}` 约定(回答者 #1)。 + +### S2 — 分发路径:修好坏的那条,让三条可发现 + +> **⚠️ 这一节被整节推翻过一次,推翻的理由比结论更重要。** +> +> 初稿在这里提出补一条 `[build] c_runtime` 轴,三值与 `cxx_runtime` 相同,其中 `host-coupled` = 把产物链到**宿主的 libc**。它已经实现并通过了全部测试,然后被整条撤销(commit 已 reset)。 +> +> **撤销理由一:那是照着 issue 提的实现方法做,不是解决 issue 的问题。** #375 的标题写着「让要分发的应用链到系统 libc」——那是提报者**已经想好的解法**。他的**问题**是「产物没法分发」。照着解法做,等于把提报者的方案当成需求。 +> +> **撤销理由二:它穿越了这个生态存在的意义所在的那条边界。** hermetic 策略明文列出禁止穿越项,第一条就是「任何 `/usr/lib*` `/lib*` 下的 `.so`(含 libc)」。xlings 是用户态发行版;一个伸手去 `/lib64` 取 libc 的产物已经不在这个发行版里了。**mcpp 能不用 host 就不用 host。** +> +> **撤销理由三:去掉 `host-coupled` 之后它不剩任何新能力。** `self-contained` 已经由 `--target x86_64-linux-musl` 表达,`toolchain-coupled` 是现状。也就是说这条轴的**全部增量就是那个不该有的值**。 +> +> 保留这段记录,是因为「同一个东西以后还会被再提一次」——下次有人拿 #375 说「加个链系统 libc 的开关吧」,这里有现成的答案。 + +**用户的真问题是「产物没法分发」,而它有三条 hermetic 答案,全都已经存在**: + +| 路径 | 命令 | libc 从哪来 | 适用 | +|---|---|---|---| +| **A. 生态闭环** | `mcpp emit xpkg` → `xlings install` | 目标机自己的 xlings 载荷,**elfpatch 在装机期重指 PT_INTERP/RUNPATH** | 目标机在生态内 | +| **B. 静态单文件** | `--target x86_64-linux-musl` | 自带,静态链接 | 任何 Linux,无任何运行期依赖 | +| **C. 自带运行时** | `mcpp pack --mode self-contained` | 自带这套工具链的 glibc + loader | 任何 Linux,含比构建机更老的 | + +**A 是这个生态真正的答案**,而且它解释了一件容易被误读的事:产物里烙的那个 `PT_INTERP` 指向构建机路径**并不构成分发障碍**——走 A 时目标机的 xlings 会重写它。#375 观察到的「目标机上路径不存在所以起不来」,前提是**绕开生态直接拷贝二进制**。 + +所以缺口不在机制,在**可发现性**,外加 **C 这条路自己是坏的**(§1.6)。 + +**要做的两件事,都不新增穿越宿主的能力:** + +1. **修 C**(§1.6 已实施):`self-contained` 的 wrapper 打坏 `/proc/self/exe`。这是三条 hermetic 路径里唯一一条自己有缺陷的,而它恰好是「目标机没有 xlings 又不想静态链接」时的那条。 +2. **让三条可发现**:`mcpp pack` 与 `mcpp build` 的文档把这三条并列写清楚,`self-contained` 在 glibc 上不可行时的诊断**逐条列出这三条**,而不是只说「不行」。一个只说「不行」的诊断会把用户推向他自己能想到的办法——而那个办法通常就是宿主 libc。 + +**明确不做**:不新增任何让 `mcpp build` 产出链宿主 libc 的开关。已有的那个决定只有一个入口(`[build] allow_host_libs` / `MCPP_ALLOW_HOST_LIBS`),再开第二个就是「同一决策两处推导」,而且这一次推导出来的是策略上被禁的那一侧。 + +> **顺带记一笔既有账**(不在本轮改):`mcpp pack --mode vendored` 是 pack 的**默认**模式,而它把 PT_INTERP 重指到 `/lib64/ld-linux-*.so.2`——即默认打包路径本身就是宿主耦合的。这与 hermetic 策略不一致,但改默认会破坏既有用户,需要单独评估。先记在这里。 + +### S3 — 环境契约(subos 一等公民) + +**mcpp 产物需要的第三样东西(configuration)由 subos 提供,mcpp 消费它,而不是自己实现。** + +三件事: + +1. **写**:mcpp 初始化 sandbox 时确保 subos 有 `subos_info` 块(今天没有,见 §1.3)。不自己造 schema——调 `xlings self init` 的对应路径,或按 schema 1 写。**权威源是 xlings,mcpp 只保证它存在。** + +2. **读并应用**:`mcpp run` / `mcpp test` 把 `subos_info.envs` 的**解析结果**(`${subosdir}` 展开后)应用到子进程,而不是今天只设 `LD_LIBRARY_PATH`(`execute.cppm:285`)。 + `LIBGL_DRIVERS_PATH` / `__EGL_VENDOR_LIBRARY_DIRS` / `XDG_DATA_DIRS` 由此**自动**到位——mcpp 侧一行图形相关的代码都不写。 + +3. **烙进分发物**:`mcpp pack` 把同一份解析结果写进 launcher —— **仅 `toolchain-coupled`**(目标机上有这个 subos);`host-coupled` 不写,因为目标机没有 subos,那里的图形栈是宿主的事。 + +**图形栈以依赖到达,不以代码到达**: + +- mcpp-index 的 `compat.glx-runtime` **废弃**,改为依赖 `xim:graphics` +- 触及 GL 的包(`compat.glfw` / `compat.opengl` / `compat.imgui` / …)声明能力需求,由 capability → xlings 包的解析落到 `xim:graphics` +- 这是 §4 那条一般原则的第一个实例:**xlings 加 Vulkan loader / 换驱动桥 / 支持新宿主形态,mcpp 侧零改动** + +--- + +## 4. 「未来 xlings 升级、mcpp 自动适配」的一般原则 + +用户的要求里最重要的一条不是修 #375,是「**以后 xlings 环境升级,mcpp 能很简单地动态适配**」。上面三条缝各自解一个症状,而让它们不再复发的是下面这条规则: + +> **凡是 xlings 拥有的事实,必须经由恰好一条数据通道到达 mcpp;mcpp 侧对该事实的再推导数必须为 0。** + +三条通道,按**生命周期**划分(不是按内容划分——这是关键,内容会变,生命周期不会): + +| 通道 | 载体 | 生命周期 | mcpp 何时读 | 承载什么 | +|---|---|---|---|---| +| **C1 载荷事实** | `/.xpkg-install.json`(需扩展) | 不可变,随版本目录 | 构建期(prepare) | loader、libdirs、abi、CRT 目录 | +| **C2 subos 事实** | `/.xlings.json` 的 `subos_info` | 可变,随 `xlings use` | 构建期取身份 + 运行期取 env | runtime 身份、env 声明 | +| **C3 宿主能力** | xlings 依赖图里的哨兵包 | 装机期探测 | **不读** | NVIDIA / WSL2 / 无 GPU | + +**C3 的正确做法是「什么都不做」**,这条值得单独说:`xim:graphics` 的哨兵机制(`pkgs/g/graphics.lua`)——每个哨兵探测一个自己不拥有的宿主侧半边,不在就「链接了零个东西然后成功返回」——意味着**「这台机器没有那个」是一个正常返回值,不是一个分支**。所以 mcpp 侧永远不需要写 `if (nvidia)`。任何时候如果 mcpp 里出现了探测宿主 GPU/图形能力的代码,它就是这条原则的违反。 + +**版本偏斜必须可检测,不能静默**:C1/C2 各带 `schema_version`。mcpp 读到不认识的高版本 → 明确降级 + 一行提示;读到缺失 → 走降级路径 + 一行提示。**沉默成功是 xlings 侧诊断出的横切属性(「『没发生』和『成功了』输出相同」),mcpp 侧不要复制它。** + +> 这里有一条来自 xlings 侧的硬教训需要照抄(`libs/sysroot.lua:38-47`):**「版本地板写成数据」永远到不了需要被告知的那些客户端**——能读这个字段的客户端恰恰是不需要被告知的那些。所以 mcpp 读 C1/C2 时,能力缺失的处置必须是**运行时可见的一行输出**,不能是一个只有新版本才会检查的字段。 + +--- + +## 5. 方案对比 + +### 5.1 mcpp 怎么拿到 xlings 的事实(§4 C1 的实现形态) + +| | A. 直接读 xlings 的磁盘文件 | B. 调 xlings CLI 查询 | **C. xlings 落一份版本化契约文件,mcpp 读** | +|---|---|---|---| +| `subos_info` | ✅ 已存在,可直接读 | 需要新子命令 | ✅ 已存在 | +| 载荷 `exports` | ❌ **没落盘** | 需要新子命令 | 需 xlings 小改(值已算出) | +| 离线 | ✅ | ⚠️ 取决于实现 | ✅ | +| 热路径开销 | 零 | ❌ 每次 prepare 起子进程 | 零 | +| 版本偏斜 | mcpp 复制 schema,易漂 | ❌ **旧 xlings 没有新子命令**,而沙箱 xlings 出了名地不刷新 | schema_version 显式 | +| 违反哪条规则 | 部分违反 R2(读端仍需知道布局) | —— | 无 | + +**推荐 C,以 A 为降级路径,B 永不进热路径。** + +理由:写端(installer)**已经**把值算全了(`installer.cppm:2394` 的注释明说「Recorded for EVERY runtime dep, declared exports or not」,正是 R1),持久化几乎零成本,而它一次性删掉 mcpp 侧四个再推导。B 在这个生态里有一个具体的、已被记录的失败模式——沙箱里的 xlings 版本长期落后于 pin,一个「新子命令」在最需要它的机器上恰好不存在。 + +**分期上这个选择是无痛的**:`subos_info`(C2)已经在盘上,所以 **S1 与 S3 的 P0 完全不依赖 xlings 改动**;只有 C1 的 `exports` 持久化需要跨仓协作,排 P1。 + +### 5.2 为什么不补 `c_runtime` 这条轴 + +这一节记录一个**被否掉的方案**,因为它已经实现过一遍,而且看起来很有说服力。 + +| | A. 补 `c_runtime`(含 `host-coupled`) | **B. 不补,修好并指明三条 hermetic 路径** | +|---|---|---| +| 解决「产物没法分发」 | ✅ | ✅ | +| 需要新概念 | 一个 manifest 键 + 一张机制表 + 一条跨层枚举镜像 | 零 | +| 是否新增穿越宿主的能力 | **是** —— 这正是它的全部增量 | 否 | +| 与 hermetic 策略 | **冲突**(禁止穿越第一条就是 `/lib*` 下的 libc) | 一致 | +| 「可不可以用宿主 libc」的回答者数 | **2**(`allow_host_libs` + 新键) | 1 | + +**选 B。** A 有三个独立的致命处,任何一个都够: + +1. **它是照着 issue 的解法做,不是解 issue 的问题**(§3-S2 撤销理由一) +2. **它穿越了这个生态存在的意义所在的边界**(理由二) +3. **去掉那个值之后它不剩任何能力**(理由三)—— 也就是说 A 列那些 ✅ 全都不是 A 独有的 + +值得单独记下:A **已经通过了全部测试** —— 12 个契约表单测、7 个渲染单测、一条覆盖五个断言的 e2e(含「默认不变」与「拒绝要出声」)。**测试全绿不能告诉你这个功能不该存在。** + +### 5.3 `toolchain-coupled` 用载荷寻址还是视图寻址 + +> **本节的推荐在评估阶段被下调过。** 初稿推荐视图寻址;查到 `doctor.cppm:369-387` 之后改为「默认不换,除非配套做完两件事」。理由如下。 + +**新证据**:mcpp 自己的 doctor **已经在检查 subos 视图悬空**,并且写下了成因: + +> Dangling symlinks under `registry/subos/default/lib` — these point into xim payload lib dirs; **a removed package leaves them broken**。 + +也就是说「视图会悬空」不是假想风险,是 mcpp 已经观测到并专门写了检查的现象。而 PT_INTERP 指向不存在的路径时,`exec` 报的是 **`No such file or directory`——指着一个明明存在的文件**,是 Linux 上最经典的假线索之一。 + +**公平地算完四格**(不能只讲对自己有利的那格): + +| 场景 | 载荷寻址(今天) | 视图寻址 | +|---|---|---| +| 载荷升级 2.39→2.44,**保留** 2.39 | ✅ | ✅(glibc 向后兼容) | +| 载荷升级 **+ GC 掉** 2.39 | ❌ 该批坏 | ✅ | +| 载荷被删(无升级) | ❌ 该批坏 | ❌ **全部坏** | +| home 迁移 | ❌ | ❌(链接仍是绝对路径) | + +**视图寻址只在第 2 格严格更好,在第 3 格的爆炸半径更差,其余两格相同。** 一格换一格,不是压倒性的。 + +| | **A. 维持载荷寻址** | B. 换视图寻址 | C. 按 role 分(bin 视图 / test 载荷) | +|---|---|---|---| +| 抗「升级+GC」 | ❌ | ✅ | ✅ | +| 悬空爆炸半径 | 小(按版本分批) | **大(全体)** | 大 | +| e2e 冲击 | 零 | ~4 个断言 PT_INTERP 的用例 | 同 B | +| 可重现性 | ✅ 完全钉死 | ⚠️ 产物行为随 subos 变,**而 fingerprint 覆盖不到**(它是 compile-side) | ⚠️ 同 B | +| 同一轴上的默认值个数 | 1 | 1 | **2** | + +**推荐 A(维持现状),除非同时做完这两件事**——做完之后再切 B: + +1. **doctor 能自动修复视图悬空**,而不只是报告(今天只 `warn`)。R3 判据:报告不是修复。 +2. **exec 失败有人话诊断**:mcpp 在 `run`/`test` 里检测到 PT_INTERP 不可达时,直接说「你的 subos 视图坏了,跑 `mcpp doctor --fix`」,而不是让用户去解读 `No such file or directory`。 + +**不推荐 C**:同一条轴上两个默认值,而「同一决策两处推导」在这个代码库里已经反复付过学费(#233/#240/#344/#336)。 + +**这条不再是「最需要拍板」的一条**——降格成机制层开关之后,它可以在 P2 独立评审,不阻塞 P0/P1。 + +--- + +## 6. 分阶段落地 + +### P0 — 零 xlings 改动 + +按「收益 ÷ 代价」排序,前两条可以立刻做且互不依赖: + +1. **文档 + `self-contained` 缺陷**(§1.6):在 `docs/02-pack-and-release.md` 写明 `/proc/self/exe` / `/proc/self/cmdline` 陷阱,并按 §3-S2 的 (i)/(ii) 择一修掉 wrapper。**这是 #375 唯一一条纯粹的既有缺陷**,不依赖本文任何架构改动。 +2. **mcpp-index**:`compat.glx-runtime` → `xim:graphics` → **#352 关闭** +3. **S3-读**:`mcpp run` / `mcpp test` 消费 `subos_info.envs`;初始化时确保 subos 有 `subos_info`(§1.3 的缺口)。第 2 条要在真实 GPU 上验成(V4),依赖这一条。 +4. **S2-可发现性**:`self-contained` 在 glibc 上不可行时的诊断逐条列出三条 hermetic 路径;`docs/02-pack-and-release.md` 与 `05-mcpp-toml.md` 并列写清 A/B/C。**不新增任何 build 期链宿主 libc 的开关。** + +### P1 — 跨仓契约 + +5. **C1**:xlings 把 `resolved_deps`/`deps_exports` 持久化(schema 版本化) +6. **S1**:`RuntimeBinding` 成为一等轴;`abi.cppm` 的 libc 维度带上版本 +7. **删除**:回答者 #1 #2 #4 #6(R3 判据:必须是**删掉**,不是**再加一条**) + +### P2 — 生态 / 需先满足前置条件 + +8. **前置**:doctor 自动修复视图悬空 + exec 失败人话诊断(§5.3 的两个前提) +9. 之后才评审:`toolchain-coupled` 是否切到视图寻址 +10. 多 glibc / platform 成员身份 —— 与 xlings 侧 platform manifest 联动(xlings 开放问题 B) + +--- + +## 7. 跨仓契约:需要 xlings 侧配合的**只有一件事** + +把 `installer.cppm` 已经算出来的记录落盘。建议形态(字段名沿用 xlings 内部已有的命名,避免第三套词汇): + +```jsonc +// /.xpkg-install.json —— 扩展,不是新文件 +{ + "schema_version": 2, + "os": "linux", "version": "2.39", "xlings_version": "2026.8.7.1", + "exports": { // 本包声明的(self_exports) + "loader": "lib64/ld-linux-x86-64.so.2", + "abi": "linux-x86_64-glibc", + "libdirs": ["lib64"] + }, + "resolved_deps": { // R1:每一个 runtime dep 都记,不只声明了的 + "xim:linux-headers@5.11.1": { + "install_dir": "…", "libdirs": ["…"], "source": "plan-exact" + } + } +} +``` + +三条要求: + +- **相对路径**(相对载荷根),不是绝对路径——绝对路径会把 home 位置烙进一个可被复制/硬链接的文件,而 mcpp 侧已经为这类问题付过学费 +- `schema_version` 必须有,且 mcpp 读到未知高版本要**降级 + 出声** +- **R1 全量**:每一个 runtime dep 都记,包括什么都没声明的那些。缺省即约定是这一族缺陷的共同上游(A2) + +**注意这不是新设计**——`.xpkg-install.json` 已存在、值已算出、字段名已定。只是把一个被丢弃的计算结果写下来。 + +--- + +## 8. 明确不做 + +- **不接管 subos 状态**。mcpp 读 subos、保证 `subos_info` 存在,**不管理** subos 生命周期、不实现 `subos use`、不写 `workspace` 版本 DB。xlings 侧已经把「recipe 里塞 build 逻辑、mcpp 反过来管 subos 状态」点名为层次未分开的症状。 +- **不在 mcpp 里探测宿主图形能力**。哨兵机制在 xlings 侧,见 §4 C3。mcpp 里出现 `if (nvidia)` 即为违规。 +- **不追 glibc 版本**。mcpp 不实现「选最新可用 libc」——那会让两台同命令的机器产出不同结果(xlings 侧对 `DEFAULT_RUNTIME` 用常量而非查表,理由相同)。 +- **不做 platform manifest**。那是 xlings 的开放问题 B,mcpp 侧只需**消费**一个 runtime 身份;platform 成员身份排 P2。 +- **不为 `/proc/self/exe` 单独设计任何东西**。见 §2.3 推论——它是 workaround 的后果,契约修好后自动消失。 +- **不动 BMI/fingerprint 的缓存身份**,除非 P2 改默认契约时另行评估。契约变化会改链接命令,**不改编译命令**,这是分期能这么切的原因。 + +--- + +## 9. 开放问题 + +- **Q1**(已在评估阶段自我下调,**不再阻塞 P0/P1**)§5.3 载荷寻址 vs 视图寻址:现推荐维持载荷寻址,先把 doctor 自动修复 + exec 失败人话诊断做完,再在 P2 独立评审是否切换。 +- **Q2**(新)`mcpp pack --mode vendored` 是 pack 的**默认**,而它把 PT_INTERP 重指到 `/lib64/ld-linux-*.so.2` —— 默认打包路径本身就是宿主耦合的,与 hermetic 策略不一致。改默认会破坏既有用户,需单独评估:改默认、打告警、还是维持并在文档里说清。 +- **Q3** 多 subos:mcpp 有 home 级 sandbox 和项目级 subos(实测 `/.xlings/subos/_/`)。S3 该读**哪一个**的 `subos_info`?倾向「构建时活动的那个」,但需要确认它在 CI 与本机的一致性。(视图寻址一旦在 P2 启用,同一个问题会变成「烙哪一个」,风险更高——这是又一条把它排到 P2 的理由。) +- **Q4** 交叉编译时 `subos_info.runtime` 描述的是宿主 subos,目标 runtime 从哪来?可能需要 `[target.].runtime` 强制显式(而不是有一个缺省)。 +- **Q5**(回答 xlings 开放问题 K)本文的答案是「**消费 subos 的身份,而不是在 subos 里跑**」——mcpp 读 `subos_info.runtime` 作为契约输入,构建仍在 mcpp 自己的 hermetic 环境里。是否与 xlings 侧的设想一致,需要跨仓确认。 +- **Q6**(需你拍板)§3-S2 的 `self-contained` wrapper:选 (i) 安装期改写 PT_INTERP(消灭问题,但 tarball 不再解开即跑),还是 (ii) 保留 wrapper + `MCPP_BUNDLE_DIR` + 文档(不破坏现有契约,但救不了第三方库)?这条**独立于**本文其余部分,可以先决先做。 + +--- + +## 10. 验证判据(每条可执行,先红后绿) + +| # | 判据 | 怎么测 | +|---|---|---| +| V1 | 三条 hermetic 路径各自在**无 mcpp** 的机器上跑通 | 干净容器里分别验:A `xlings install` 后跑;B musl 产物原样拷进去跑;C `pack --mode self-contained` 解开就跑。**必须是真实干净容器** —— 在 mcpp 沙箱里跑测不出东西,那里载荷路径恰好总是存在 | +| V2 | **每一个** pack 模式产出的程序,`/proc/self/exe` 都指向自身 | 产物内打印 `readlink("/proc/self/exe")` 并断言 == 自身路径,**四个模式全跑**。今天 `self-contained` 必红(§1.6),这条就是它的先红后绿 | +| V3 | (仅当 P2 切视图寻址)产物在载荷升级+GC 后仍可运行 | 装 glibc@2.44、`xlings use`、**删掉 2.39**,不重新构建直接跑旧产物 | +| V4 | GL 程序在 hermetic 图形栈下起窗口 | `xlings install graphics` 后跑 GLFW e2e;**断言渲染器不是 llvmpipe**——「跑起来了」是假绿,#352 的教训是要问「谁答的」 | +| V5 | mcpp 侧 loader/libdir 推导点数量单调下降 | 对 §2.1 六个位置做静态计数,CI 守住上界 | +| V6 | subos 无 `subos_info` 时**出声降级** | 删掉块,断言 stderr 有一行,且构建仍成功 | +| V7 | 产物不引用任何 `/usr/lib*` `/lib*` 下的 `.so` | 对 `mcpp build` 产物做 `ldd` 闭包扫描,断言无宿主路径命中(`allow_host_libs` 显式开启时豁免)| + +> **两条来自本仓历史的验证陷阱,必须避开**: +> - **CI 全绿不等于覆盖**(#346:18 个 job 全绿也没测到大链接)。V1/V3 必须在**真实的干净容器**里跑,不能只在 CI 的 mcpp 沙箱里跑——那里 PT_INTERP 恰好总是存在。 +> - **「有输出」不是判据**(#352 的 exit 255 无输出;`VS_VERSION_INFO` 全文搜索是假判据)。V4 必须断言**渲染器身份**,不是断言窗口出现。 + +--- + +## 11. 多维评估 + +本节的每个数字都在本机量过。**结论先行:P0 的三条(文档/wrapper、图形栈、S3-读)是低风险高收益,可以直接做;S2 中等;S1 是唯一有实质回归风险的一条,排 P1 是对的;P2 目前不该做。** + +### 11.1 实现代价(量化) + +| 项 | 数字 | 依据 | +|---|---|---| +| ~~`c_runtime` 实现面~~ | ~~9 文件~~ | **已撤销**(§5.2)。实测过:9 文件 / `flags.cppm` 约 30 处 / 12+7 单测 / 1 条 e2e 全绿 —— 记在这里是为了说明「代价可控」从来不是做不做的判据 | +| **BMI / 对象缓存失效** | **零** | fingerprint 是 **compile-side 10 字段**(`fingerprint.cppm:3-8`),不含链接侧;`target//` 目录名不变,只重链 | +| e2e 需改 | ~4 个(203 个中) | 断言 PT_INTERP 的:`30_pack_modes` `86_llvm_hermetic_link` `28_target_static` `168_build_mcpp_musl_host_static` | +| e2e 耦合载荷路径 | 58 处 / 26 文件 | 大多是工具链解析,不是 PT_INTERP;S1 落地时才受影响 | +| mcpp-index 破坏面 | **3 个包 / 81** | `compat.glfw` `compat.glx-headers` `compat.vulkan-runtime` | + +**「零缓存失效」是这套方案最大的成本优势**,和 #336 当年便宜的原因相同:契约只改链接命令。 + +**净增 / 净删**:新增 1 个 manifest 键 + 1 张 mechanism 表 + 1 个 RuntimeBinding 类型 + 2 条读文件路径;删除 §2.1 六个回答者里的 4–5 个,外加整个 `compat.glx-runtime`(约 130 行宿主探测 Lua)。满足 R3。 + +### 11.2 用户使用层 + +| 用户 | 影响 | +|---|---| +| **普通用户**(build/run/test) | P0 全部**零感知**;S3 让 GL 程序从「exit 255 无输出」变成能跑,是纯增益 | +| **分发者** | `pack --mode self-contained` 从「静默打坏 exe 相对资源解析」变成可用(§1.6),这是三条 hermetic 路径里唯一一条自己有缺陷的;另外两条(A 生态闭环 / B musl 静态)本来就能用,缺的是文档 | +| **库作者** | `compat.glx-runtime` 废弃是破坏性变更,需过渡期(保留空壳 provider 一个版本) | + +**新增认知负担:零。** 不加键、不加轴、不加词汇。用户模型仍是「target × pack mode」,只是三条分发路径终于被并列写出来了。这是选 B 而非 A 的直接收益之一。 + +### 11.3 稳定性 + +**新增失败模式三个,两个可控、一个未受控:** + +| 失败模式 | 处置 | 评价 | +|---|---|---| +| 读 `subos_info` 失败 / 缺失 | 出声降级(V6) | 可控 | +| xlings schema 演进 | `schema_version` 显式 + 降级出声 | 可控 | +| **视图寻址下产物行为随 subos 变化** | **fingerprint 覆盖不到**(compile-side) | **未受控** —— 这是把它降级到 P2 并加两个前置条件的直接原因(§5.3) | + +**P0/P1 本身不引入新的运行期失败模式**:契约层只是把已有机制(pack 的 patchelf)提前到链接期,产出的 PT_INTERP 形状是 `30_pack_modes.sh` 今天已经在断言的那些。 + +### 11.4 跨平台(最弱的一维,必须直说) + +| 平台 | 分发路径可用性 | S1 runtime 身份 | S3 env / 图形 | +|---|---|---|---| +| Linux / glibc | **3** | `glibc@X`,有意义 | ✅ 全部收益 | +| Linux / musl | 1(恒 self-contained) | `musl@X` | 部分 | +| **macOS** | **1** —— libSystem 恒为宿主(`platform/common.cppm:110` `supports_full_static = is_linux`) | 近乎常量 | **无**(无 mesa/glvnd) | +| **Windows** | 2(静/动 CRT),且与 `linkage` 共用 `-static` 拼写 | 近乎常量 | **无** | + +**收益 ≈ 全在 Linux。** macOS 上这条轴是退化的——只有一个合法值。 + +macOS/Windows 上没有 mesa/glvnd,所以 S3 的图形收益是 Linux-only;而分发路径 A/B/C 里,B(musl 静态)本身就是 Linux 概念。**这是诚实的不对称,不是退化** —— 因为本轮不新增任何全平台的轴,也就不存在「为一个平台的问题给三个平台加概念」。 + +若 review 认为 Linux-only 收益不值一条全局轴,替代是放进 `[target.'cfg(linux)']`。**我不推荐**——那会引入第二套作用域规则,而 `cxx_runtime` 已经确立了「全局轴 + 平台退化」的先例。 + +### 11.5 简洁优雅 + +**正面**:零新词汇(三值抄 `cxx_runtime`,runtime binding 抄 xlings);三层模型已存在,新轴是**填满一个已知空格**而非加一个维度;§4 的三通道按**生命周期**而非内容划分,所以 xlings 加新能力时通道数不变。 + +**负面(必须记账)**: +- 概念总数确实上升,§11.2 已列。 +- **初稿的 `subos-coupled` 第四值破坏了对称性**——这是评估阶段发现并已修正的一处真实设计缺陷(§3-S2 的框)。修正之后词汇仍是三个,且「视图 vs 载荷」回到它本来的层(Mechanism)。**如果没做这次评估,这条会带着一个多余的契约值进入实施。** + +### 11.6 兼容性 + +| 面 | 影响 | 风险 | +|---|---|---| +| 已构建产物 | 不受影响(载荷目录仍在) | 无 | +| `[pack].default_mode` / tarball 后缀 frozen wire format | **不动** | 无 | +| 老 subos 无 `subos_info` | 降级 + 提示 | 低 | +| 老 xlings 无持久化 exports | 走 §5.1 的 A 降级路径;P1 才依赖 | 低 | +| `compat.glx-runtime` 下游 | 3 个包 | **低于预期**(初判以为是索引级风险,实测是叶子簇) | +| **新 mcpp + 老 xlings** | 必须验 | **中** —— 已知「沙箱 xlings 长期落后于 pin」,这一格最容易假绿 | + +### 11.7 总评 + +| 维度 | P0(wrapper 修复 + 图形栈 + S3-读) | S2(分发路径可发现性) | S1(RuntimeBinding) | ~~c_runtime~~ / P2 视图寻址 | +|---|---|---|---|---| +| 实现代价 | 低 | 中 | 中高(动 `abi.cppm`,参与依赖解析) | 低 | +| 用户收益 | **高**(#352 从不可用变可用) | 中高 | 低(内部收敛) | 低 | +| 稳定性风险 | 低 | 低 | **中**(ABI 维度回归) | **高**(未受控) | +| 跨平台价值 | Linux-only | Linux 高 / Win 中 / mac 退化 | 全平台(收敛推导) | Linux-only | +| 兼容性风险 | 低(3 个包) | 低 | 中 | 中 | +| **建议** | **直接做** | **做** | **P1,单独评审** | **暂不做** | diff --git a/.agents/docs/2026-08-08-xlings-runtime-substrate-implementation-plan.md b/.agents/docs/2026-08-08-xlings-runtime-substrate-implementation-plan.md new file mode 100644 index 00000000..954ad6de --- /dev/null +++ b/.agents/docs/2026-08-08-xlings-runtime-substrate-implementation-plan.md @@ -0,0 +1,840 @@ +# xlings 运行时底座 —— 实施计划 + +> **For agentic workers:** 逐 task 执行,每个 task 自带 red→green→commit 循环。 + +**Goal:** 让 mcpp 把 xlings 当作可查询的运行时底座:补上 libc 轴的分发契约(`c_runtime`)、消费 subos 的运行时身份与环境声明、修掉 `self-contained` 打包模式自产的 `/proc/self/exe` 陷阱。 + +**Architecture:** 三条缝。S2 在既有 `mcpp.build.distribution` 的三层模型(Role→Contract→Mechanism)上**复用词汇、新增 libc 机制表**,由 `linkmodel` 单点渲染;S3 新增 `mcpp.xlings.subos_info` 读取 xlings 的 `subos_info` 块,`run`/`test` 应用其 env;S1 让 runtime 身份成为显式轴。P2(视图寻址)不做。 + +**Tech Stack:** C++23 modules,gtest(`tests/unit/`),shell e2e(`tests/e2e/`),nlohmann::json(`mcpp.libs.json`)。 + +设计文档:`.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md` + +## Global Constraints + +- **词汇不新造**:契约三值 `self-contained` | `toolchain-coupled` | `host-coupled`,与 `[build] cxx_runtime` 逐字相同。runtime binding 拼写 `@`,与 xlings `subos_info.runtime` 逐字相同。 +- **默认值不变**:`c_runtime` 缺省 = `toolchain-coupled` = 今天的行为。引入这条轴本身**不改变任何现有产物**。 +- **Mechanism 必须是 total function**:每格要么给 flags,要么 `degraded=true` + 非空 `diagnostic`。静默无操作是禁止的结果。 +- **不引入新的宿主探测**:mcpp 里不得出现探测 GPU / 图形能力的代码。 +- **schema 缺失/未知版本必须出声**:一行 stderr,构建继续。 +- **P2 不做**:`toolchain-coupled` 保持载荷寻址,不切视图寻址。 +- **平台特殊处理下沉**:平台差异放 `src/platform/` 对应模块,不在 `build/` 里写 `#ifdef`。 + +--- + +### Task 1: `self-contained` 打包模式的 `/proc/self/exe` 陷阱 + +**Files:** +- Modify: `src/pack/pack.cppm:431-446`(`write_bundle_all_wrappers`) +- Modify: `docs/02-pack-and-release.md`(§ Mode `self-contained` 之后) +- Test: `tests/e2e/30_pack_modes.sh` + +**Interfaces:** +- Consumes: 无 +- Produces: 分发包契约新增环境变量 `MCPP_BUNDLE_DIR`(bundle 根目录绝对路径),由 `run.sh` / 顶层同名脚本导出。 + +**背景**:ELF 规范禁止 `PT_INTERP` 用 `$ORIGIN`,所以 `self-contained` 只能经 `ld.so --library-path` 启动;代价是内核把 `/proc/self/exe` 指向 loader、`/proc/self/cmdline` 混入 `--library-path`。本 task 不消除这个约束(那需要安装期改写 PT_INTERP,见设计文档 Q6 选项 i),而是**把它变成一个有声明、可编程的契约**,并写进文档。 + +- [ ] **Step 1: 写失败的 e2e 断言** + +在 `tests/e2e/30_pack_modes.sh` 的 Mode B(bundle-all)段落后追加: + +```sh +# self-contained: the wrapper must export MCPP_BUNDLE_DIR so an application +# can resolve resources despite /proc/self/exe pointing at the loader. +grep -q 'MCPP_BUNDLE_DIR' "$TMP/b/myapp-0.1.0-x86_64-linux-gnu-bundle-all/run.sh" || { + echo "Mode B: run.sh does not export MCPP_BUNDLE_DIR"; exit 1; } +out=$("$TMP/b/myapp-0.1.0-x86_64-linux-gnu-bundle-all/run.sh" 2>&1) || true +echo "$out" | grep -q 'Hello' || { echo "Mode B: wrapper broke execution: $out"; exit 1; } +``` + +- [ ] **Step 2: 跑,确认失败** + +```bash +bash tests/e2e/30_pack_modes.sh +``` +Expected: FAIL,`run.sh does not export MCPP_BUNDLE_DIR` + +- [ ] **Step 3: 改 wrapper** + +`src/pack/pack.cppm` 的 `write_bundle_all_wrappers` body 改为: + +```cpp + auto body = std::format( + "#!/bin/sh\n" + "# Auto-generated by `mcpp pack --mode self-contained`. Launches the\n" + "# bundled binary through the bundled dynamic linker so the package\n" + "# is portable across glibc versions.\n" + "#\n" + "# TRAP, and why this variable exists: launching through the loader\n" + "# makes the kernel set /proc/self/exe to the LOADER, not to the\n" + "# program, and /proc/self/cmdline carries --library-path. Any\n" + "# \"find my resources next to the executable\" logic silently\n" + "# resolves against the loader's directory instead. MCPP_BUNDLE_DIR\n" + "# is the answer that survives: resolve against it first and fall\n" + "# back to /proc/self/exe only when it is unset.\n" + "here=$(cd \"$(dirname \"$0\")\" && pwd)\n" + "MCPP_BUNDLE_DIR=\"$here\"; export MCPP_BUNDLE_DIR\n" + "exec \"$here/lib/{}\" --library-path \"$here/lib\" \"$here/bin/{}\" \"$@\"\n", + loaderName, binaryName); +``` + +- [ ] **Step 4: 跑,确认通过** + +```bash +bash tests/e2e/30_pack_modes.sh +``` +Expected: PASS + +- [ ] **Step 5: 文档** + +在 `docs/02-pack-and-release.md` 的 self-contained loader 说明之后追加一节: + +```markdown +#### Trap: `/proc/self/exe` under the bundled loader + +Launching through the loader means the kernel sets `/proc/self/exe` to the +**loader**, not to your program, and `/proc/self/cmdline` carries the +`--library-path` argument. Every "find my resources next to the executable" +path silently resolves against the wrong directory — GUI toolkits looking for +fonts or `assets/`, and helper binaries shipped alongside the program. + +The wrapper exports `MCPP_BUNDLE_DIR` (the bundle root) for this. Resolve +against it first: + +```c +const char *base = getenv("MCPP_BUNDLE_DIR"); /* set by run.sh */ +/* fall back to /proc/self/exe only when unset */ +``` + +If your application cannot be changed, use `--mode vendored` instead: it +repoints `PT_INTERP` at the host loader, so `/proc/self/exe` is correct — at +the cost of requiring the host's glibc to be at least as new as the one the +artifact was built against. +``` + +同一节的中文版加到 `docs/zh/` 下对应文件(若存在)。 + +- [ ] **Step 6: Commit** + +```bash +git add src/pack/pack.cppm docs/02-pack-and-release.md tests/e2e/30_pack_modes.sh +git commit -m "fix(pack): the self-contained wrapper broke /proc/self/exe and never said so" +``` + +--- + +### ~~Task 2–4: `c_runtime` 契约轴~~ —— 已撤销 + +**做过,全绿,然后整条 reset。** 设计文档 §3-S2 与 §5.2 记录了理由;摘要: + +- 它是照着 #375 提报者**提出的解法**做的,不是解他的**问题**(问题是「产物没法分发」) +- 它唯一的新能力是 `host-coupled` = 把产物链到**宿主 libc**,而 hermetic 策略禁止穿越的第一条就是 `/lib*` 下的 libc。**mcpp 能不用 host 就不用 host** +- 去掉那个值之后这条轴不剩任何能力(`self-contained` 已由 `--target ...-musl` 表达) +- 它会造出「可不可以用宿主 libc」的**第二个回答者**(既有的是 `[build] allow_host_libs`) + +**代替它的是 Task 4'(下)**:#375 的真问题有三条 hermetic 答案,全都已存在;缺口在「其中一条自己是坏的」(Task 1 已修)加「三条都不可发现」。 + +--- + +### Task 4': 让三条 hermetic 分发路径可发现 + +**Files:** +- Modify: `docs/02-pack-and-release.md` + `docs/zh/02-pack-and-release.md` +- Modify: `docs/00-getting-started.md` 或 `10-publishing-a-library.md`(择一放「怎么分发」的入口) +- Test: 文档改动,无自动化断言;判据是 §10 的 V1(三条路径各自在干净容器里跑通) + +**要写清的三条,并列、等价、都不用宿主 libc:** + +| 路径 | 命令 | libc 从哪来 | 什么时候选它 | +|---|---|---|---| +| A. 生态闭环 | `mcpp emit xpkg` → `xlings install` | 目标机自己的 xlings 载荷,**elfpatch 装机期重指 PT_INTERP/RUNPATH** | 目标机在生态内 | +| B. 静态单文件 | `--target x86_64-linux-musl` | 自带,静态 | 任何 Linux,零运行期依赖 | +| C. 自带运行时 | `mcpp pack --mode self-contained` | 自带这套工具链的 glibc + loader | 任何 Linux,含比构建机更老的 | + +**A 要特别写明一件容易误读的事**:产物里烙的 `PT_INTERP` 指向构建机路径**不构成分发障碍** —— 走 A 时目标机的 xlings 会重写它。#375 观察到的「路径不存在所以起不来」,前提是绕开生态直接拷贝二进制。 + +**不写**:任何「让产物链到系统 libc」的做法。已有的那个决定只有一个入口(`[build] allow_host_libs`),文档不给它第二个说法。 + +- [ ] **Step 1** 在 `docs/02-pack-and-release.md` 顶部「Two axes」之前加一节 "Three ways to ship, none of which use the host's libc",内容如上表 +- [ ] **Step 2** 中文版同步 +- [ ] **Step 3** Commit + +--- + +### Task 5: 读 xlings 的 subos 自描述 + +**Files:** +- Create: `src/xlings/subos_info.cppm`(模块 `mcpp.xlings.subos_info`) +- Test: `tests/unit/test_subos_info.cpp` + +**Interfaces:** +- Consumes: `mcpp.libs.json` +- Produces: + ```cpp + namespace mcpp::xlings::subos { + inline constexpr int kSupportedSchema = 1; + struct EnvDecl { std::string var, op, value; }; + struct Provider { std::string binding; std::vector decls; }; + struct Info { + int schema = 0; + std::string runtime; // "glibc@2.39" + std::vector providers; // sorted by binding + bool present = false; // the block existed at all + std::string note; // non-empty ⇒ caller MUST print it + }; + Info read(const std::filesystem::path& subosDir); + std::string family_of(std::string_view runtime, std::string_view arch = "x86_64"); + std::vector> + resolve_env(const Info&, const std::filesystem::path& subosDir); + } + ``` + +- [ ] **Step 1: 写失败的单测** + +创建 `tests/unit/test_subos_info.cpp`: + +```cpp +#include + +import std; +import mcpp.xlings.subos_info; + +namespace { + +namespace su = mcpp::xlings::subos; + +struct Tmp { + std::filesystem::path dir; + Tmp() { + dir = std::filesystem::temp_directory_path() + / ("mcpp_subos_" + std::to_string(std::random_device{}())); + std::filesystem::create_directories(dir); + } + ~Tmp() { std::error_code ec; std::filesystem::remove_all(dir, ec); } + void write(std::string_view body) { + std::ofstream(dir / ".xlings.json") << body; + } +}; + +TEST(SubosInfo, ReadsRuntimeAndEnvs) { + Tmp t; + t.write(R"({ + "workspace": {}, + "subos_info": { + "schema_version": 1, + "runtime": "glibc@2.39", + "envs": [ + { "binding": "mesa@25.0.7.1", "decls": [ + { "var": "LIBGL_DRIVERS_PATH", "op": "prepend", + "value": "${subosdir}/usr/lib/dri" } + ]} + ] + } + })"); + auto info = su::read(t.dir); + EXPECT_TRUE(info.present); + EXPECT_EQ(info.schema, 1); + EXPECT_EQ(info.runtime, "glibc@2.39"); + ASSERT_EQ(info.providers.size(), 1u); + ASSERT_EQ(info.providers[0].decls.size(), 1u); + EXPECT_EQ(info.providers[0].decls[0].var, "LIBGL_DRIVERS_PATH"); + EXPECT_TRUE(info.note.empty()); +} + +// ${subosdir} is expanded against the subos this block was read from. +TEST(SubosInfo, ResolvesSubosdirPlaceholder) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39", + "envs":[{"binding":"mesa@1","decls":[ + {"var":"LIBGL_DRIVERS_PATH","op":"prepend","value":"${subosdir}/usr/lib/dri"}]}]}})"); + auto env = su::resolve_env(su::read(t.dir), t.dir); + ASSERT_EQ(env.size(), 1u); + EXPECT_EQ(env[0].first, "LIBGL_DRIVERS_PATH"); + EXPECT_EQ(env[0].second, (t.dir / "usr/lib/dri").string()); +} + +// A subos made before xlings grew the block: degrade, and SAY SO. Silence +// here is what made mcpp#352 hard to find in the first place. +TEST(SubosInfo, MissingBlockDegradesWithANote) { + Tmp t; + t.write(R"({"workspace":{}})"); + auto info = su::read(t.dir); + EXPECT_FALSE(info.present); + EXPECT_TRUE(info.runtime.empty()); + EXPECT_FALSE(info.note.empty()); +} + +// A schema newer than we understand: use what we can, and say we are behind. +TEST(SubosInfo, NewerSchemaDegradesWithANote) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":99,"runtime":"glibc@2.44","envs":[]}})"); + auto info = su::read(t.dir); + EXPECT_TRUE(info.present); + EXPECT_EQ(info.runtime, "glibc@2.44"); + EXPECT_FALSE(info.note.empty()); +} + +TEST(SubosInfo, NoFileAtAllIsNotACrash) { + Tmp t; // nothing written + auto info = su::read(t.dir); + EXPECT_FALSE(info.present); + EXPECT_FALSE(info.note.empty()); +} + +// The family mapping must agree with xlings's own, verbatim. +TEST(SubosInfo, FamilyOfMirrorsXlings) { + EXPECT_EQ(su::family_of("glibc@2.39"), "linux-x86_64-glibc"); + EXPECT_EQ(su::family_of("musl@1.2.5"), "linux-x86_64-musl"); + EXPECT_EQ(su::family_of("glibc@2.39", "aarch64"), "linux-aarch64-glibc"); + EXPECT_EQ(su::family_of("wasi-libc@1"), "wasm32-wasi"); + EXPECT_EQ(su::family_of("nonsense@1"), "unknown"); +} + +// Malformed JSON must not take the build down. +TEST(SubosInfo, MalformedJsonDegrades) { + Tmp t; + t.write("{ this is not json"); + auto info = su::read(t.dir); + EXPECT_FALSE(info.present); + EXPECT_FALSE(info.note.empty()); +} + +} // namespace +``` + +- [ ] **Step 2: 跑,确认失败** + +```bash +mcpp test --filter SubosInfo +``` +Expected: FAIL,模块不存在 + +- [ ] **Step 3: 实现** + +创建 `src/xlings/subos_info.cppm`: + +```cpp +// mcpp.xlings.subos_info — read the `subos_info` block xlings writes into a +// subos's own `.xlings.json`. +// +// WHAT THIS IS FOR +// +// A program needs three things: bootstrap (PT_INTERP + CRT + libc), discovery +// (PATH + RPATH), and configuration (env vars). xlings had the first two — +// glibc + elfpatch, xvm + shims — and until it grew this block, nothing for +// the third. Its own module comment names the consequence: mcpp#352, a GLFW +// binary that links fine and exits 255 because nothing told it where the GL +// drivers are. +// +// mcpp is the consumer of the third one. This module ONLY reads and resolves; +// it never writes the block and never manages subos lifecycle — that is +// xlings's layer, and mcpp reaching into it is the layering inversion the +// ecosystem design calls out by name. +// +// A note on silence: every degradation here fills `note`, and callers are +// required to print it. "It did not happen" and "it succeeded" producing the +// same output is the property that made #352 expensive to find. +// +// Design: .agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md §3-S3 + +export module mcpp.xlings.subos_info; + +import std; +import mcpp.libs.json; + +export namespace mcpp::xlings::subos { + +// The schema this build understands. A higher one on disk is usable — we read +// the fields we know — but the caller is told it is reading a newer format. +inline constexpr int kSupportedSchema = 1; + +inline constexpr std::string_view kBlock = "subos_info"; + +struct EnvDecl { + std::string var; + std::string op; // "set" | "prepend" + std::string value; // may contain ${subosdir} +}; + +struct Provider { + std::string binding; // "@" + std::vector decls; +}; + +struct Info { + int schema = 0; + std::string runtime; // "glibc@2.39" + std::vector providers; // sorted by binding + bool present = false; + // Non-empty ⇒ the caller MUST surface it. Never a hard error: a missing + // or newer block degrades the experience, it does not invalidate a build. + std::string note; +}; + +// The runtime string is self-describing: "glibc@2.39" says Linux/glibc +// without a second field that could disagree with it. Mirrors xlings's +// `subos::manifest::family_of` — the mapping is small, stable and part of the +// cross-repo contract, so it is asserted in tests rather than left implicit. +std::string family_of(std::string_view runtime, + std::string_view arch = "x86_64") { + const auto at = runtime.find('@'); + const auto name = runtime.substr(0, at == std::string_view::npos + ? runtime.size() : at); + if (name == "glibc") return std::format("linux-{}-glibc", arch); + if (name == "musl") return std::format("linux-{}-musl", arch); + if (name == "wasi-libc") return "wasm32-wasi"; + if (name == "macos_sdk") return std::format("darwin-{}", arch); + if (name == "ucrt") return std::format("windows-{}-ucrt", arch); + return "unknown"; +} + +Info read(const std::filesystem::path& subosDir) { + Info info; + auto path = subosDir / ".xlings.json"; + std::error_code ec; + if (!std::filesystem::exists(path, ec)) { + info.note = std::format( + "subos '{}' has no .xlings.json, so it cannot say which runtime it " + "is or what environment its programs need", subosDir.string()); + return info; + } + std::ifstream is(path); + auto doc = nlohmann::json::parse(is, nullptr, /*allow_exceptions=*/false); + if (doc.is_discarded() || !doc.is_object()) { + info.note = std::format("subos manifest {} is not readable JSON", + path.string()); + return info; + } + auto it = doc.find(std::string(kBlock)); + if (it == doc.end() || !it->is_object()) { + info.note = std::format( + "subos '{}' does not describe itself (no `{}` block): its runtime " + "identity and environment declarations are unavailable. A newer " + "xlings writes this block; `xlings self update` adds it", + subosDir.string(), kBlock); + return info; + } + info.present = true; + if (auto v = it->find("schema_version"); + v != it->end() && v->is_number_integer()) + info.schema = v->get(); + if (auto v = it->find("runtime"); v != it->end() && v->is_string()) + info.runtime = v->get(); + + if (auto envs = it->find("envs"); envs != it->end() && envs->is_array()) { + for (auto& p : *envs) { + if (!p.is_object()) continue; + Provider prov; + if (auto b = p.find("binding"); b != p.end() && b->is_string()) + prov.binding = b->get(); + if (auto ds = p.find("decls"); ds != p.end() && ds->is_array()) { + for (auto& d : *ds) { + if (!d.is_object()) continue; + EnvDecl e; + if (auto x = d.find("var"); x != d.end() && x->is_string()) + e.var = x->get(); + if (auto x = d.find("op"); x != d.end() && x->is_string()) + e.op = x->get(); + if (auto x = d.find("value"); x != d.end() && x->is_string()) + e.value = x->get(); + if (!e.var.empty()) prov.decls.push_back(std::move(e)); + } + } + info.providers.push_back(std::move(prov)); + } + } + // Sorted by binding, matching xlings's own ordering, so two reads of the + // same subos produce the same environment in the same order. + std::ranges::sort(info.providers, + [](auto const& a, auto const& b) { return a.binding < b.binding; }); + + if (info.schema > kSupportedSchema) + info.note = std::format( + "subos '{}' declares schema {}, newer than the {} this mcpp " + "understands; reading the fields it knows and ignoring the rest", + subosDir.string(), info.schema, kSupportedSchema); + return info; +} + +// Resolve the declarations into concrete (var, value) pairs, `${subosdir}` +// expanded. `prepend` entries for the same variable are joined with ':' in +// provider order; a `set` replaces whatever came before it, which is xlings's +// own precedence. +std::vector> +resolve_env(const Info& info, const std::filesystem::path& subosDir) { + std::vector> out; + auto expand = [&](std::string v) { + constexpr std::string_view kPh = "${subosdir}"; + for (auto pos = v.find(kPh); pos != std::string::npos; + pos = v.find(kPh, pos)) + v.replace(pos, kPh.size(), subosDir.string()); + return v; + }; + for (auto const& p : info.providers) { + for (auto const& d : p.decls) { + auto value = expand(d.value); + auto hit = std::ranges::find(out, d.var, &std::pair::first); + if (hit == out.end()) { out.emplace_back(d.var, value); continue; } + if (d.op == "set") { hit->second = value; continue; } + // prepend, de-duplicated: a doubled entry must not accumulate + // across nested invocations. + if (hit->second != value + && !hit->second.starts_with(value + ":") + && hit->second.find(":" + value) == std::string::npos) + hit->second = value + ":" + hit->second; + } + } + return out; +} + +} // namespace mcpp::xlings::subos +``` + +- [ ] **Step 4: 跑,确认通过** + +```bash +mcpp test --filter SubosInfo +``` +Expected: PASS(7 组全绿) + +- [ ] **Step 5: Commit** + +```bash +git add src/xlings/subos_info.cppm tests/unit/test_subos_info.cpp +git commit -m "feat(xlings): read the subos's own description instead of inferring it" +``` + +--- + +### Task 6: `mcpp run` / `mcpp test` 应用 subos 环境 + +**Files:** +- Modify: `src/build/plan.cppm`(把解析好的 env 放进 plan) +- Modify: `src/build/execute.cppm:279-300, 1210-1230`(应用到子进程) +- Test: `tests/e2e/196_subos_env_reaches_program.sh`(新建) + +**Interfaces:** +- Consumes: `mcpp::xlings::subos::{read, resolve_env}` +- Produces: `BuildPlan` 新增 `std::vector> subosEnv;` + +**为什么不是自己拼图形变量**:`LIBGL_DRIVERS_PATH` / `__EGL_VENDOR_LIBRARY_DIRS` / `XDG_DATA_DIRS` 的值由 xlings 的图形包声明,mcpp 只是把它们传下去。mcpp 里出现任何图形相关的字面量都是这条设计的违反。 + +- [ ] **Step 1: 写失败的 e2e** + +创建 `tests/e2e/196_subos_env_reaches_program.sh`: + +```sh +#!/usr/bin/env bash +# requires: linux +# A variable a package declared into the subos must reach the program `mcpp +# run` launches. Until this landed, subos env declarations were applied only +# by `xlings subos use`, so a program started by mcpp saw none of them — the +# reason a GL binary could link fine and exit 255 (mcpp#352). +set -euo pipefail +. "$(dirname "$0")/_common.sh" + +proj=$(mktemp -d) +trap 'rm -rf "$proj"' EXIT + +# A subos that declares one variable, in xlings's own schema. +subos="$proj/subos" +mkdir -p "$subos/usr/lib/dri" +cat > "$subos/.xlings.json" <<'EOF' +{ "workspace": {}, + "subos_info": { "schema_version": 1, "runtime": "glibc@2.39", + "envs": [ { "binding": "probe@1", "decls": [ + { "var": "MCPP_E2E_PROBE", "op": "prepend", + "value": "${subosdir}/usr/lib/dri" } ] } ] } } +EOF + +cd "$proj" +"$MCPP" new hello >/dev/null +cd hello +cat > src/main.cpp <<'EOF' +#include +#include +int main() { + const char* v = std::getenv("MCPP_E2E_PROBE"); + std::printf("PROBE=%s\n", v ? v : "(unset)"); +} +EOF + +out=$(MCPP_SUBOS_DIR="$subos" "$MCPP" run 2>&1) +echo "$out" | grep -q "PROBE=$subos/usr/lib/dri" || { + echo "subos env did not reach the program:"; echo "$out"; exit 1; } +echo "PASS: subos env reaches the program" +``` + +`chmod +x tests/e2e/196_subos_env_reaches_program.sh` + +> `MCPP_SUBOS_DIR` 是本 task 引入的**测试与覆盖用**入口:默认解析仍是活动 subos。它让这条 e2e 不需要改动用户真实环境——记住 memory 里那条教训:破坏性脚本不在真机上试。 + +- [ ] **Step 2: 跑,确认失败** + +```bash +bash tests/e2e/196_subos_env_reaches_program.sh +``` +Expected: FAIL,`PROBE=(unset)` + +- [ ] **Step 3: 实现** + +`src/build/plan.cppm`,在 `runtimeLibraryDirs` 那段之后: + +```cpp + // The subos's own environment declarations. mcpp reads them and passes + // them on; it does not author them and does not know what they mean. That + // is the whole point: when xlings adds a Vulkan loader or a new driver + // bridge, this code does not change. + { + auto subosDir = mcpp::xlings::subos_dir_for_build(cfg); + auto info = mcpp::xlings::subos::read(subosDir); + if (!info.note.empty()) mcpp::ui::warning(info.note); + plan.subosEnv = mcpp::xlings::subos::resolve_env(info, subosDir); + } +``` + +`src/xlings.cppm` 加一个解析器(单点,不让每个消费者各猜一次): + +```cpp + // The subos whose environment a program built here should run under. + // + // MCPP_SUBOS_DIR overrides it outright — that exists so tests can exercise + // this path without touching the developer's real environment, and so a + // user can point a run at another subos without switching the active one. + std::filesystem::path subos_dir_for_build(const config::GlobalConfig& cfg); +``` + +`src/build/execute.cppm`:在构造子进程环境的两处(`run` 的 279-300 与 ninja 侧 1210-1230)把 `plan.subosEnv` 合并进去,`prepend` 语义与既有 `prepend_path_list` 一致。 + +- [ ] **Step 4: 跑,确认通过** + +```bash +bash tests/e2e/196_subos_env_reaches_program.sh +``` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/build src/xlings.cppm tests/e2e/196_subos_env_reaches_program.sh +git commit -m "feat(run): a program mcpp launches gets its subos's environment" +``` + +--- + +### Task 7: 运行时身份成为显式轴 + +**Files:** +- Modify: `src/toolchain/abi.cppm`(libc 维度带版本) +- Modify: `src/toolchain/model.cppm`(`Toolchain` 加 `runtimeBinding`) +- Modify: `src/build/prepare.cppm`(解析优先级) +- Test: `tests/unit/test_abi.cpp` + +**Interfaces:** +- Consumes: `subos::read(...).runtime`、`subos::family_of` +- Produces: `AbiProfile` 新增 `std::string libcVersion;`(空 = 未知,**不参与匹配**),`Toolchain::runtimeBinding` + +**收窄的实现范围与理由**:`abi.cppm` 的 `libc` 维度参与依赖解析,给它加一个**参与匹配**的版本轴会立刻改变全索引的解析结果(memory 里 `index-floor-must-degrade` 是同一族事故)。所以本 task 只做两件事:①把版本**记录**下来并在 `mcpp doctor` / `--verbose` 里可见;②`runtimeBinding` 的解析优先级落地。**匹配语义不变**,留给 platform manifest 那一轮。 + +- [ ] **Step 1: 写失败的单测** + +在 `tests/unit/test_abi.cpp` 追加: + +```cpp +// The libc dimension gains a version, and it is RECORDED, not matched. A +// dependency that says `abi:glibc` must keep matching a glibc@2.39 toolchain +// exactly as it did before — changing that would re-resolve the whole index. +TEST(Abi, LibcVersionIsRecordedButNotMatched) { + mcpp::toolchain::Toolchain tc; + tc.targetTriple = "x86_64-linux-gnu"; + tc.runtimeBinding = "glibc@2.39"; + auto p = mcpp::toolchain::abi_profile(tc); + EXPECT_EQ(p.libc, "glibc"); + EXPECT_EQ(p.libcVersion, "2.39"); + + auto c = mcpp::toolchain::parse_abi_capability("abi:glibc"); + ASSERT_TRUE(c.has_value()); + EXPECT_TRUE(mcpp::toolchain::satisfies(p, *c)); +} + +// No binding resolved: the dimension still answers, without a version. +TEST(Abi, LibcVersionEmptyWithoutABinding) { + mcpp::toolchain::Toolchain tc; + tc.targetTriple = "x86_64-linux-gnu"; + auto p = mcpp::toolchain::abi_profile(tc); + EXPECT_EQ(p.libc, "glibc"); + EXPECT_TRUE(p.libcVersion.empty()); +} +``` + +- [ ] **Step 2: 跑,确认失败** + +```bash +mcpp test --filter Abi +``` +Expected: FAIL,`runtimeBinding` / `libcVersion` 不存在 + +- [ ] **Step 3: 实现** + +`model.cppm` 的 `Toolchain` 加: + +```cpp + // The runtime this toolchain's output is built against, in xlings's own + // spelling ("glibc@2.39"). Resolved in prepare, in this order, every step + // explicit — a "default is the convention" step is what grows a second + // answerer (see the design doc §S1): + // 1. --runtime + // 2. [target.].runtime / [build].runtime + // 3. the active subos's `subos_info.runtime` + // 4. the payload actually resolved, with a note + std::string runtimeBinding; +``` + +`abi.cppm` 的 `AbiProfile` 加 `std::string libcVersion;`,并在 `abi_profile()` 的两条路径里从 `tc.runtimeBinding` 的 `@` 之后取值。**`satisfies()` 不动。** + +`prepare.cppm` 按上述四级解析并写进 `tc.runtimeBinding`;第 4 级发一条 `ui::info`。 + +- [ ] **Step 4: 跑,确认通过** + +```bash +mcpp test --filter Abi +``` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add src/toolchain src/build/prepare.cppm tests/unit/test_abi.cpp +git commit -m "feat(toolchain): the runtime a build targets is read, not inferred" +``` + +--- + +### Task 8: 文档 + +**Files:** +- Modify: `docs/05-mcpp-toml.md`(`c_runtime`) +- Modify: `docs/02-pack-and-release.md`(contract × mode 的关系) +- Modify: `docs/03-toolchains.md`(runtime binding) +- Modify: `docs/zh/` 下的对应文件 +- Modify: `.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md`(标注已实施) + +- [ ] **Step 1: `05-mcpp-toml.md` 加 `c_runtime`** + +紧邻既有 `cxx_runtime` 一节,写明三值、默认值、以及与 `mcpp pack --mode` 的函数关系表(设计文档 §3-S2 那张)。**必须写清 `host-coupled` 的语义是「宿主 glibc ≥ 构建时的那份」**,不是「任何 Linux」。 + +- [ ] **Step 2: `02-pack-and-release.md` 加一节 "Contract vs mode"** + +说明 mode 管「带多少东西」、contract 管「承诺什么」,并给出不可分发契约在 pack 时会被拒绝的行为。 + +- [ ] **Step 3: `03-toolchains.md` 加 runtime binding** + +说明四级解析顺序与 `--runtime`。 + +- [ ] **Step 4: 校验中文版同步** + +```bash +ls docs/zh/ +``` +对每个改过的英文文档,同步中文版。 + +- [ ] **Step 5: Commit** + +```bash +git add docs .agents/docs +git commit -m "docs: the two runtime contracts, and what each promises" +``` + +--- + +### Task 9: 版本 + xlings pin + PR + +**Files:** +- Modify: `src/version.cppm` +- Modify: `src/xlings.cppm`(`kXlingsVersion`) + +- [ ] **Step 1: 查最新 xlings 版本** + +```bash +gh api repos/openxlings/xlings/releases/latest --jq .tag_name +``` + +- [ ] **Step 2: bump 两个常量** + +`MCPP_VERSION` → `2026.8.8.1`;`kXlingsVersion` → 上一步的值。 + +> **只动这两个。** bootstrap pin(`.xlings.json` 那组)是自举起点,不随发布走 —— 一起 bump 会让全部 CI 去装一个还不存在的版本。 + +- [ ] **Step 3: 机器校验 pin 一致** + +```bash +.github/tools/check_version_pins.sh +``` +Expected: 通过 + +- [ ] **Step 4: 全量本地验证** + +```bash +mcpp build && mcpp test +for t in 30_pack_modes 195_c_runtime_host_coupled 196_subos_env_reaches_program; do + bash "tests/e2e/$t.sh" || echo "FAIL: $t" +done +``` + +- [ ] **Step 5: 开 PR** + +```bash +git checkout -b feat/xlings-runtime-substrate +git push -u origin feat/xlings-runtime-substrate +gh pr create --title "feat: xlings 作为运行时底座 —— libc 分发契约 + subos 环境 (#375, #352)" --body-file .agents/docs/pr-body.md +``` + +--- + +### Task 10: mcpp-index 图形栈迁移(独立仓、独立 PR) + +**Files(`/home/speak/workspace/github/mcpplibs/mcpp-index`):** +- Modify: `pkgs/c/compat.glfw.lua`, `pkgs/c/compat.glx-headers.lua`, `pkgs/c/compat.vulkan-runtime.lua` +- Deprecate: `pkgs/c/compat.glx-runtime.lua` + +- [ ] **Step 1: 确认下游只有三个** + +```bash +cd /home/speak/workspace/github/mcpplibs/mcpp-index && grep -rln "glx-runtime" pkgs/ +``` +Expected: 恰好 4 个文件(含它自己) + +- [ ] **Step 2: 三个消费者改为依赖 `xim:graphics`** + +`compat.glx-runtime` 保留为**空壳 provider**(仍声明 `provides = {"opengl.glx.driver","x11.display"}`,但不再 symlink 宿主库),一个版本之后再删——下游包的 manifest 已发布,不能立刻失效(与 `index-floor-must-degrade` 同一条教训)。 + +- [ ] **Step 3: 验** + +```bash +bash tests/smoke.sh 2>/dev/null || ls tests/ +``` +按该仓既有验证入口跑。**判据是渲染器身份,不是「窗口出现了」。** + +- [ ] **Step 4: PR** + +--- + +## Self-Review + +**Spec coverage** — 设计文档各节 → task 映射: + +| 设计文档 | Task | +|---|---| +| §1.6 / Q6 `self-contained` wrapper | T1 | +| §3-S2 `c_runtime` 契约 | T2 + T3 + T4 | +| §3-S3 subos 环境 | T5 + T6 | +| §3-S1 运行时身份 | T7 | +| §1.5 图形栈迁移 | T10 | +| §6 P0 文档 | T1 Step5 + T8 | +| §7 跨仓契约(xlings 落盘 exports) | **不在本 PR** —— 需 xlings 侧改动,设计文档已排 P1 尾;本 PR 的 §5.1「A 降级路径」足以工作 | +| §5.3 / P2 视图寻址 | **不做**(设计文档明确) | + +**未覆盖且是有意的**:§2.1 六个回答者里,本 PR 收敛的是 #2/#3/#5(loader 选择统一到 `link_tokens` + `distro_loader_path`)。#1/#4/#6 依赖 xlings 侧落盘 exports,留 P1。V5 的静态计数守的是「不再增加」,不是「已经归零」。 + +**Type consistency** — 跨 task 一致性已核:`dist::Contract` / `dist::Role` / `dist::Format` 三者在 T2/T4 同名;`libcdist::Addressing` 与 `tc::LibcAddressing` 是**有意的镜像对**(层次不倒置),T3 的注释与 T2 的测试都点明了这一点,并由 T4 的 `static_cast` 单点转换。`subos::Info` / `resolve_env` 在 T5 定义、T6 消费,签名一致。 diff --git a/docs/02-pack-and-release.md b/docs/02-pack-and-release.md index e57b870d..a493820b 100644 --- a/docs/02-pack-and-release.md +++ b/docs/02-pack-and-release.md @@ -1,9 +1,42 @@ # 02 — Packaging for Release -> A default dynamically linked binary produced by `mcpp build` normally has a -> loader and RUNPATH tied to the build sandbox. To distribute it to other -> machines or deploy it to a server, use `mcpp pack` to produce a release -> tarball or directory with the appropriate runtime closure. +> A default dynamically linked binary produced by `mcpp build` has a loader and +> RUNPATH tied to the build sandbox. It is a development artifact, not a +> deliverable. Three routes turn it into one — and none of them uses the host's +> C library. + +## Three ways to ship + +Every route below produces an artifact whose C runtime comes from the +ecosystem, never from `/lib64`. That is deliberate: mcpp builds against a +private glibc precisely so a binary's behaviour does not depend on which +distribution happens to be underneath it, and reaching back out to the host's +libc to distribute would give that away at the last step. + +| | Route | Command | Where its C runtime comes from | Choose it when | +|---|---|---|---|---| +| **A** | Through the ecosystem | `mcpp emit xpkg` → `xlings install ` | the target machine's own xlings payloads | the target has xlings | +| **B** | One static file | `mcpp build --target x86_64-linux-musl` | nowhere — it is linked in | you want a single file with no runtime at all | +| **C** | Carry the runtime | `mcpp pack --mode self-contained` | shipped inside the bundle | any Linux, including older than the build machine | + +**On route A, and the thing that surprises people.** The `PT_INTERP` baked into +a freshly built binary points at *your* machine's payload, so copying that file +to another machine by hand does not work — the path is not there. That is not a +property of the artifact so much as of the copy: installed through `xlings`, the +package's ELF files are repointed at the target machine's own payloads at +install time. The baked path is a build-machine detail, not a distribution +format. If you are hand-copying binaries between machines, you want B or C. + +**On route B.** `--target …-musl` implies a static link, so there is no loader, +no RUNPATH and nothing to find at run time. It is the smallest and most +portable result, and the one to reach for first when the program does not need +glibc-specific behaviour (NSS lookups, `dlopen` of host plugins). + +**On route C.** The bundle carries this toolchain's glibc and its loader, so it +runs on distributions older than the build machine — the case B cannot cover +when glibc is actually required. Read the `/proc/self/exe` note below before +choosing it: launching through a bundled loader changes what the program sees +about itself. ## Two axes: target (libc) × mode (bundling depth) @@ -128,6 +161,37 @@ exec "$here/lib/ld-linux-x86-64.so.2" --library-path "$here/lib" "$here/bin/myap The layout and wrapper above use an x86_64 example. The packer derives the loader name from the target; for aarch64 it is `ld-linux-aarch64.so.1`. +#### Trap: `/proc/self/exe` under the bundled loader + +Being started *by* the loader has a consequence the layout above does not +show: the kernel sets `/proc/self/exe` to the **loader**, not to your program, +and `/proc/self/cmdline` carries the `--library-path` argument. Every "find my +resources next to the executable" path therefore resolves against `lib/` +instead of the bundle root — and it does so silently. In practice that means +a GUI toolkit rendering blank text because it cannot find its fonts, an +`assets/` directory that appears to be missing, and helper binaries shipped +alongside the program that cannot be located. Code that parses `argv` from +`/proc/self/cmdline` sees the loader's arguments mixed in. + +This affects `self-contained` only. `vendored`, `system` and `static` all +carry a `PT_INTERP` that the kernel can use directly, so `/proc/self/exe` is +correct there. + +The wrapper exports **`MCPP_BUNDLE_DIR`** (the bundle root) for this. Resolve +against it first and fall back only when it is unset: + +```c +const char *base = getenv("MCPP_BUNDLE_DIR"); /* set by run.sh */ +if (!base) { + /* not launched through the wrapper — /proc/self/exe is trustworthy */ +} +``` + +If the application cannot be changed — a third-party GUI framework doing its +own resolution, say — use `--mode vendored` instead. It repoints `PT_INTERP` +at the host loader, at the cost of requiring the host's glibc to be at least +as new as the one the artifact was built against. + ## Configuration Packaging behavior is configured via the `[pack]` section in `mcpp.toml`. The diff --git a/docs/zh/02-pack-and-release.md b/docs/zh/02-pack-and-release.md index fe7e285a..e6eaff0a 100644 --- a/docs/zh/02-pack-and-release.md +++ b/docs/zh/02-pack-and-release.md @@ -1,8 +1,34 @@ # 02 — 发布打包 -> 默认的动态链接 `mcpp build` 产物通常会把 loader 与 RUNPATH 指向构建沙盒。 -> 如需分发至其他机器或部署至服务器,应使用 `mcpp pack` 生成带有适当运行时闭包的 -> 发布 tarball 或目录。 +> 默认的动态链接 `mcpp build` 产物会把 loader 与 RUNPATH 指向构建沙盒。它是 +> 开发产物,不是交付物。有三条路把它变成交付物 —— **没有一条使用宿主的 C 库**。 + +## 三种分发方式 + +下面每一条产出的产物,其 C 运行时都来自生态,而不是 `/lib64`。这是有意的: +mcpp 之所以针对私有 glibc 构建,正是为了让产物的行为不取决于底下是哪个发行版; +如果最后一步又伸手去拿宿主的 libc,前面这件事就白做了。 + +| | 方式 | 命令 | C 运行时来自哪里 | 何时选它 | +|---|---|---|---|---| +| **A** | 走生态 | `mcpp emit xpkg` → `xlings install ` | 目标机自己的 xlings 载荷 | 目标机装了 xlings | +| **B** | 静态单文件 | `mcpp build --target x86_64-linux-musl` | 不来自任何地方 —— 已链进去 | 想要一个无任何运行时依赖的单文件 | +| **C** | 自带运行时 | `mcpp pack --mode self-contained` | 随 bundle 一起分发 | 任何 Linux,含比构建机更老的 | + +**关于 A,以及那个让人意外的地方。** 刚构建出的二进制里烙的 `PT_INTERP` 指向 +**你这台机器**的载荷,所以手工把这个文件拷到另一台机器上跑不起来——那个路径 +在那边不存在。与其说这是产物的性质,不如说是「手工拷贝」这个动作的性质:经 +`xlings` 安装时,包里的 ELF 会在**装机期被重指到目标机自己的载荷**。烙进去的 +路径是构建机的细节,不是分发格式。如果你就是要在机器之间手工拷二进制,那你要 +的是 B 或 C。 + +**关于 B。** `--target …-musl` 隐含静态链接,所以没有 loader、没有 RUNPATH、 +运行期不需要找任何东西。它的结果最小也最可移植,在程序不需要 glibc 专有行为 +(NSS 查询、`dlopen` 宿主插件)时应当首选。 + +**关于 C。** bundle 里带着这套工具链的 glibc 与 loader,因此能在比构建机更老的 +发行版上跑 —— 这是 B 覆盖不了、而又确实需要 glibc 时的那一格。选它之前先读下面 +的 `/proc/self/exe` 一节:经 bundled loader 启动会改变程序对「自己在哪」的认知。 ## 两条轴:target(libc) × mode(打包深度) @@ -120,6 +146,32 @@ exec "$here/lib/ld-linux-x86-64.so.2" --library-path "$here/lib" "$here/bin/myap 上面的布局与 wrapper 以 x86_64 为例。打包器会按 target 推导 loader 名称；aarch64 对应 `ld-linux-aarch64.so.1`。 +#### 陷阱:经 bundled loader 启动后的 `/proc/self/exe` + +「由 loader 启动」有一个上面的布局看不出来的后果:内核会把 `/proc/self/exe` +指向 **loader**,而不是你的程序;`/proc/self/cmdline` 里也混进了 +`--library-path`。于是所有「在可执行文件旁边找资源」的逻辑都会解析到 `lib/` +而不是 bundle 根目录——而且是**静默**的。实际表现是:GUI 框架找不到字体因而 +文字渲染空白、`assets/` 目录看起来不存在、随包分发的辅助二进制定位失败。 +按 `/proc/self/cmdline` 解析 argv 的代码则会拿到混入 loader 参数的结果。 + +这只影响 `self-contained`。`vendored`、`system`、`static` 的 `PT_INTERP` +都能被内核直接使用,`/proc/self/exe` 是正确的。 + +wrapper 为此导出 **`MCPP_BUNDLE_DIR`**(bundle 根目录)。优先用它,只在未设置 +时回退: + +```c +const char *base = getenv("MCPP_BUNDLE_DIR"); /* 由 run.sh 设置 */ +if (!base) { + /* 不是经 wrapper 启动的 —— 此时 /proc/self/exe 可信 */ +} +``` + +如果应用本身改不了(比如第三方 GUI 框架自己做解析),改用 `--mode vendored`: +它把 `PT_INTERP` 重指到宿主 loader,`/proc/self/exe` 正常,代价是要求宿主 +glibc 不低于构建时所用的那份。 + ## 配置项 打包行为通过 `mcpp.toml` 中的 `[pack]` 节配置,常用字段如下: diff --git a/mcpp.toml b/mcpp.toml index e0c17b61..d2a6dbb6 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.7.1" +version = "2026.8.8.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 6a0e9b8c..b4cb5c1e 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -20,6 +20,8 @@ import mcpp.manifest; import mcpp.modgraph.scanner; import mcpp.toolchain.stdmod; import mcpp.xlings; +import mcpp.xlings.subos_info; +import mcpp.log; import mcpp.platform; import mcpp.fetcher.progress; import mcpp.project; @@ -60,6 +62,12 @@ struct BuildCacheEntry { // plan.runtimeLibraryDirs is empty. std::string runEnvKey; std::string runEnvValue; + // The subos this build's toolchain belongs to (mcpp#352). The DIRECTORY, + // never the resolved variables: the environment is the subos's property + // and must be re-read on every run, while WHICH subos is the build's + // property and would otherwise be unknowable on the fast path -- which + // has no toolchain to derive it from. + std::string subosDir; // The resolved profile this entry was built for. Entries used to be keyed // by target triple alone, and the fast paths only refuse to run when an // EXPLICIT --profile/--dev/--release is passed — so a bare `mcpp build` @@ -139,6 +147,16 @@ std::vector read_build_cache(const std::filesystem::path& proje std::getline(f, e.runEnvValue); haveNextLine = static_cast(std::getline(f, line)); } + // Optional subos line. Same back-compat contract: absent ⇒ empty ⇒ + // the run fast path treats the entry as a miss, exactly as it already + // does for a cache written before runtimeEnvKey existed. Running with + // a DIFFERENT environment than the full path would be worse than not + // using the cache at all -- the program would work once and then + // silently stop finding its runtime data. + if (haveNextLine && line.starts_with("subos=")) { + e.subosDir = line.substr(6); + haveNextLine = static_cast(std::getline(f, line)); + } // Optional profile line. Same back-compat contract as the two blocks // above: absent ⇒ e.profile stays empty ⇒ every fast path treats the // entry as a miss and falls through to prepare_build. @@ -167,7 +185,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, const std::string& runEnvKey = "", const std::string& runEnvValue = "", const std::string& profile = "", - const std::string& cacheMode = "") { + const std::string& cacheMode = "", + const std::string& subosDir = "") { auto path = projectRoot / kBuildCacheFile; auto entries = read_build_cache(projectRoot); @@ -182,7 +201,7 @@ void write_build_cache(const std::filesystem::path& projectRoot, // Insert at front (MRU). BuildCacheEntry newEntry{targetTriple, outputDir.string(), ninjaProgram, fingerprintHex, runtimeEnvKey, runtimeEnvValue, std::move(runTargets), - runEnvKey, runEnvValue, profile, cacheMode}; + runEnvKey, runEnvValue, subosDir, profile, cacheMode}; entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -210,6 +229,7 @@ void write_build_cache(const std::filesystem::path& projectRoot, for (auto& [name, exe] : e.runTargets) f << name << '\t' << exe << '\n'; f << "runEnv=" << e.runEnvKey << '\n'; f << e.runEnvValue << '\n'; + f << "subos=" << e.subosDir << '\n'; f << "profile=" << e.profile << '\n'; f << "cacheMode=" << e.cacheMode << '\n'; } @@ -287,6 +307,55 @@ compute_run_env(const mcpp::build::BuildPlan& plan) { return {key, value}; } +// The environment the active subos declares for the programs it hosts +// (mcpp#352). +// +// A GL application needs three things and mcpp only ever supplied two: the +// binary links (bootstrap), it finds its libraries (RPATH), and then it has to +// be told which driver module to load and which GL vendors exist. That third +// one is a set of environment variables, xlings's graphics packages declare +// them into the subos, and until now nothing carried them to a program mcpp +// launched — `xlings subos use` applied them, `mcpp run` did not. Hence a +// binary that links fine and exits 255 with no output. +// +// Resolved at RUN time, deliberately not cached with the build: these values +// belong to the subos, not to the build, and a user who switches subos between +// `mcpp build` and `mcpp run` must get the new one. It is a file read. +// +// mcpp does not know what any of these variables MEAN, and that is the design: +// when the ecosystem gains a Vulkan loader or a new driver bridge, the +// declaration changes and this code does not. +// The subos a RUN should use: an explicit override if the caller set one, +// otherwise the subos this build belongs to. +// +// The override lives HERE and not in the derivation, because the derivation's +// answer is cached and this one must not be: MCPP_SUBOS_DIR says "for this +// invocation". It exists so tests can exercise this path without touching a +// developer's real subos — an earlier e2e wrote through a symlink and +// permanently broke a real toolchain — and so a user can point one run at +// another subos without switching the active one. +std::filesystem::path subos_dir_for_run(const std::filesystem::path& buildSubos) { + if (const char* e = std::getenv("MCPP_SUBOS_DIR"); e && *e) + return std::filesystem::path(e); + return buildSubos; +} + +std::vector> +compute_subos_env(const mcpp::build::BuildPlan& plan) { + auto built = mcpp::xlings::paths::subos_dir_of(plan.toolchain.binaryPath); + auto dir = subos_dir_for_run(built ? *built : std::filesystem::path{}); + if (dir.empty()) return {}; + auto info = mcpp::xlings::subos::read(dir); + // The note is a `verbose` line rather than a warning: a subos with no + // self-description is the normal state of every machine whose subos + // predates the block, and a warning on every run would train people to + // ignore it. It becomes loud only where it explains a failure — the GL + // diagnostic path in doctor. + if (!info.note.empty()) + mcpp::log::verbose("subos", info.note); + return mcpp::xlings::subos::resolve_env(info, dir); +} + // Compile a prepared BuildContext. Shared between `mcpp build` and `mcpp run` // so the latter doesn't call prepare_build twice (and re-print the toolchain // resolution banner). @@ -398,12 +467,14 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, auto fpHex = ctx.outputDir.filename().string(); auto runTargets = compute_run_targets(ctx.plan); auto [runEnvKey, runEnvValue] = compute_run_env(ctx.plan); + auto subosDir = mcpp::xlings::paths::subos_dir_of(ctx.plan.toolchain.binaryPath); write_build_cache(ctx.projectRoot, ctx.outputDir, r->ninjaProgram, std::string(targetOverride), fpHex, r->runtimeEnvKey.empty() ? "-" : r->runtimeEnvKey, r->runtimeEnvValue, std::move(runTargets), runEnvKey, runEnvValue, - ctx.profile, std::string(cache_mode_name(ctx.cacheMode))); + ctx.profile, std::string(cache_mode_name(ctx.cacheMode)), + subosDir ? subosDir->string() : std::string{}); } // The one place the --strict policy is settled. Degradations reported by @@ -754,6 +825,25 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, std::vector> childEnv; if (!match->runEnvKey.empty() && !match->runEnvValue.empty()) childEnv.emplace_back(match->runEnvKey, match->runEnvValue); + // ...and the subos's declared environment, re-READ here rather than taken + // from the cache. Which subos is a build property (cached above); what it + // declares is the subos's own, and a user who installs a graphics stack + // between two runs must get it without rebuilding. + // + // This is the half that a fast path is most likely to lose, and losing it + // would be invisible in the worst way: the first `mcpp run` after a build + // takes the full path and works, every later one takes this path and does + // not. A GL program would run once and then stop finding its driver. + { + // Same rule as the full path, through the same helper: an override + // for this invocation, else the subos this build was recorded against. + auto subosDir = subos_dir_for_run(std::filesystem::path(match->subosDir)); + if (!subosDir.empty()) { + auto info = mcpp::xlings::subos::read(subosDir); + for (auto& kv : mcpp::xlings::subos::resolve_env(info, subosDir)) + childEnv.push_back(std::move(kv)); + } + } return mcpp::platform::process::run_exec(argv, childEnv) == 0 ? 0 : 1; } @@ -827,6 +917,8 @@ export int build_run_target(const std::optional& targetName, auto [runEnvKey, runEnvValue] = compute_run_env(ctx->plan); if (!runEnvKey.empty() && !runEnvValue.empty()) childEnv.emplace_back(runEnvKey, runEnvValue); + // ...plus whatever the subos declares for the programs it hosts (#352). + for (auto& kv : compute_subos_env(ctx->plan)) childEnv.push_back(std::move(kv)); // Direct exec (no /bin/sh): the loader env reaches ONLY the target child, // never mcpp or a host shell. Fixes the bundled-glibc-vs-host-libtinfo @@ -1213,6 +1305,9 @@ export int run_tests(std::span passthrough, auto runtimeEnvKey = mcpp::platform::env::runtime_library_path_key(); auto runtimeEnvValue = mcpp::platform::env::prepend_path_list( runtimeEnvKey, ctx->plan.runtimeLibraryDirs); + // Read once for the whole run rather than per test: it is one file, and + // every test in a run belongs to the same subos. + const auto subosEnv = compute_subos_env(ctx->plan); // macOS deliberately has no runtime-library-path key (env.cppm): injecting // DYLD_LIBRARY_PATH would reach every executable ninja launches and can @@ -1283,6 +1378,10 @@ export int run_tests(std::span passthrough, std::vector> childEnv; if (!runtimeEnvKey.empty() && !runtimeEnvValue.empty()) childEnv.emplace_back(runtimeEnvKey, runtimeEnvValue); + // ...and the subos's declared environment, same as `mcpp run` (#352). + // A GL test that cannot find a driver fails the same way a GL program + // does, so it must be told the same things. + for (auto& kv : subosEnv) childEnv.push_back(kv); // Prepend the sandbox's subos/default/bin to the CHILD PATH so test // binaries that shell out to bootstrapped tools (patchelf, ninja) find diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 6323cf54..c41866a6 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -434,10 +434,27 @@ write_bundle_all_wrappers(const std::filesystem::path& stagingRoot, { auto body = std::format( "#!/bin/sh\n" - "# Auto-generated by `mcpp pack --mode bundle-all`. Launches the\n" + "# Auto-generated by `mcpp pack --mode self-contained`. Launches the\n" "# bundled binary through the bundled dynamic linker so the package\n" "# is fully portable across glibc versions.\n" + "#\n" + "# WHY MCPP_BUNDLE_DIR EXISTS -- the trap this launch has\n" + "#\n" + "# The ELF spec forbids $ORIGIN in PT_INTERP, so a bundle that carries\n" + "# its own loader has to be started BY that loader. The kernel then\n" + "# sets /proc/self/exe to the loader, not to the program, and\n" + "# /proc/self/cmdline carries --library-path. Every \"find my resources\n" + "# next to the executable\" path therefore resolves against lib/ --\n" + "# silently: fonts and assets are simply not found, and a helper binary\n" + "# shipped alongside the program is not found either.\n" + "#\n" + "# This variable is the answer that survives. Resolve against it first\n" + "# and fall back to /proc/self/exe only when it is unset. Applications\n" + "# that cannot be changed should use `--mode vendored` instead, where\n" + "# PT_INTERP is the host loader and /proc/self/exe is correct.\n" "here=$(cd \"$(dirname \"$0\")\" && pwd)\n" + "MCPP_BUNDLE_DIR=\"$here\"\n" + "export MCPP_BUNDLE_DIR\n" "exec \"$here/lib/{}\" --library-path \"$here/lib\" \"$here/bin/{}\" \"$@\"\n", loaderName, binaryName); if (auto r = write_executable_script(stagingRoot / "run.sh", body); !r) return r; diff --git a/src/version.cppm b/src/version.cppm index 86b874ae..bd258c4a 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.7.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.8.1"; } // namespace mcpp diff --git a/src/xlings.cppm b/src/xlings.cppm index 873b50a1..e05f9457 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -74,6 +74,26 @@ namespace paths { std::optional xpkgs_from_compiler(const std::filesystem::path& compilerBin); + // The subos a TOOLCHAIN belongs to (mcpp#352). + // + // Derived from the compiler rather than from a global: a build already + // knows which home its toolchain came from, and asking a second source + // would let the two disagree — the "one question, several answerers" shape + // that the same investigation found four times over on the xlings side. + // + // A PURE derivation, with no environment override in it. That is not an + // omission: this value gets written into the build cache, and an override + // means "for this invocation", not "for this build from now on". Caching + // one would make a single `MCPP_SUBOS_DIR=… mcpp run` silently change + // where every later run looked. Which subos a RUN should use is a + // different question, answered in execute.cppm. + // + // Empty when the toolchain is not sandbox-resident — a system compiler is + // the user's explicit choice of the host world, and there is no subos + // speaking for it. + std::optional + subos_dir_of(const std::filesystem::path& compilerBin); + // Find a sibling xim tool relative to a compiler binary. // e.g. find_sibling_tool(gcc_bin, "binutils") returns highest version // dir of xim-x-binutils. @@ -673,6 +693,19 @@ xpkgs_from_compiler(const std::filesystem::path& compilerBin) { return std::nullopt; } +std::optional +subos_dir_of(const std::filesystem::path& compilerBin) { + auto xpkgs = xpkgs_from_compiler(compilerBin); + if (!xpkgs) return std::nullopt; + // /data/xpkgs → /subos/default. Spelled from the xpkgs dir + // rather than from Env so a toolchain inherited from ANOTHER home resolves + // to that home's subos, which is the one whose payloads the binary was + // actually linked against. + auto home = xpkgs->parent_path().parent_path(); + if (home.empty()) return std::nullopt; + return home / "subos" / "default"; +} + std::optional find_sibling_tool(const std::filesystem::path& compilerBin, std::string_view tool) { diff --git a/src/xlings/subos_info.cppm b/src/xlings/subos_info.cppm new file mode 100644 index 00000000..0054fb2b --- /dev/null +++ b/src/xlings/subos_info.cppm @@ -0,0 +1,242 @@ +// mcpp.xlings.subos_info — read the `subos_info` block xlings writes into a +// subos's own `.xlings.json`. +// +// WHAT THIS IS FOR +// +// A program needs three things to run: bootstrap (PT_INTERP + CRT + libc), +// discovery (PATH + RPATH), and configuration (env vars). xlings had the +// first two — glibc + elfpatch, xvm + shims — and until it grew this block, +// nothing for the third. Its own module comment names the consequence by +// number: mcpp#352, a GLFW binary that links fine and exits 255 because +// nothing told it where the GL drivers are. +// +// mcpp is the consumer of the third one. A program mcpp launches gets the +// environment its subos declares, so when the ecosystem gains a Vulkan loader +// or a new driver bridge, this file does not change — the declaration does. +// That is the whole point of reading rather than knowing: mcpp must never +// contain the string "LIBGL_DRIVERS_PATH", because the moment it does, the +// graphics stack has two owners. +// +// SCOPE: read and resolve. This module never WRITES the block and never +// manages subos lifecycle — that is xlings's layer, and mcpp reaching into it +// is the layering inversion the three-tier ecosystem design calls out by name +// ("recipe 里塞 build 逻辑、mcpp 反过来管 subos 状态" — both symptoms of the +// same confusion). +// +// A NOTE ON SILENCE +// +// Every degradation here fills `note`, and callers are required to print it. +// "It did not happen" and "it succeeded" producing identical output is the +// property that made #352 expensive, and a subos with no block is not an +// exotic case: mcpp's own sandbox subos was measured in exactly that state, +// with 356 workspace entries and no self-description at all. +// +// Design: .agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md §3-S3 + +export module mcpp.xlings.subos_info; + +import std; +import mcpp.libs.json; +import mcpp.platform; + +export namespace mcpp::xlings::subos { + +// The schema this build understands. A HIGHER one on disk is still read — we +// take the fields we know and say so. Refusing outright would let a newer +// xlings break an older mcpp, which is the failure shape the index-floor +// incident already paid for once: publishing data must not invalidate the +// program that reads it. +inline constexpr int kSupportedSchema = 1; + +inline constexpr std::string_view kBlock = "subos_info"; + +struct EnvDecl { + std::string var; + std::string op; // "set" | "prepend" + std::string value; // may contain ${subosdir} +}; + +struct Provider { + std::string binding; // "@" + std::vector decls; +}; + +struct Info { + int schema = 0; + std::string runtime; // "glibc@2.39" + std::vector providers; // sorted by binding + bool present = false; + // Non-empty ⇒ the caller MUST surface it. Never an error: a missing or + // newer block degrades the experience; it does not invalidate a build. + std::string note; +}; + +// The runtime string is self-describing: "glibc@2.39" says Linux/glibc +// without a second field that could disagree with it. +// +// Mirrors xlings's `subos::manifest::family_of`. Two implementations of one +// mapping is a cost, and the alternative — asking the xlings binary — costs +// a subprocess in the build's hot path and fails on exactly the machines +// where it matters (a sandbox xlings that has not been updated). The mapping +// is five rows and stable, and test_subos_info.cpp pins every one of them, so +// a drift is a test failure rather than a silent ABI disagreement. +std::string family_of(std::string_view runtime, + std::string_view arch = "x86_64") { + const auto at = runtime.find('@'); + const auto name = runtime.substr(0, at == std::string_view::npos + ? runtime.size() : at); + if (name == "glibc") return std::format("linux-{}-glibc", arch); + if (name == "musl") return std::format("linux-{}-musl", arch); + if (name == "wasi-libc") return "wasm32-wasi"; + if (name == "macos_sdk") return std::format("darwin-{}", arch); + if (name == "ucrt") return std::format("windows-{}-ucrt", arch); + return "unknown"; +} + +Info read(const std::filesystem::path& subosDir) { + Info info; + auto path = subosDir / ".xlings.json"; + std::error_code ec; + if (!std::filesystem::exists(path, ec)) { + info.note = std::format( + "subos '{}' has no .xlings.json, so it cannot say which runtime it " + "is or what environment its programs need", + subosDir.string()); + return info; + } + + std::ifstream is(path); + auto doc = nlohmann::json::parse(is, nullptr, /*allow_exceptions=*/false); + if (doc.is_discarded() || !doc.is_object()) { + info.note = std::format("subos manifest {} is not readable JSON", + path.string()); + return info; + } + + auto it = doc.find(std::string(kBlock)); + if (it == doc.end() || !it->is_object()) { + info.note = std::format( + "subos '{}' does not describe itself (no `{}` block), so programs " + "run from here get no environment it declares — a GL application " + "will not find its drivers. A newer xlings writes this block; " + "`xlings self update` adds it", + subosDir.string(), kBlock); + return info; + } + + info.present = true; + if (auto v = it->find("schema_version"); + v != it->end() && v->is_number_integer()) + info.schema = v->get(); + if (auto v = it->find("runtime"); v != it->end() && v->is_string()) + info.runtime = v->get(); + + if (auto envs = it->find("envs"); envs != it->end() && envs->is_array()) { + for (auto const& p : *envs) { + if (!p.is_object()) continue; + Provider prov; + if (auto b = p.find("binding"); b != p.end() && b->is_string()) + prov.binding = b->get(); + if (auto ds = p.find("decls"); ds != p.end() && ds->is_array()) { + for (auto const& d : *ds) { + if (!d.is_object()) continue; + EnvDecl e; + if (auto x = d.find("var"); x != d.end() && x->is_string()) + e.var = x->get(); + if (auto x = d.find("op"); x != d.end() && x->is_string()) + e.op = x->get(); + if (auto x = d.find("value"); x != d.end() && x->is_string()) + e.value = x->get(); + // A declaration with no variable name is not a partial + // declaration to be guessed at — it is malformed input, + // and the right thing is to leave it out rather than + // invent a name for it. + if (!e.var.empty()) prov.decls.push_back(std::move(e)); + } + } + info.providers.push_back(std::move(prov)); + } + } + + // Sorted by binding, matching xlings's own ordering, so two reads of one + // subos produce the same environment in the same order. Ordering is not + // cosmetic here: it decides which provider wins a list variable, and + // libglvnd resolves GL vendors by exactly that order. + std::sort(info.providers.begin(), info.providers.end(), + [](Provider const& a, Provider const& b) { + return a.binding < b.binding; + }); + + if (info.schema > kSupportedSchema) + info.note = std::format( + "subos '{}' declares schema {}, newer than the {} this mcpp " + "understands; reading the fields it knows and ignoring the rest", + subosDir.string(), info.schema, kSupportedSchema); + return info; +} + +// Resolve the declarations into concrete (var, value) pairs with +// `${subosdir}` expanded. +// +// `prepend` joins in provider order and de-duplicates; `set` replaces. That +// is xlings's own precedence, and the de-duplication matters because these +// variables are inherited: without it a nested invocation grows the list +// every time. +// +// The separator is the PLATFORM's, never a literal ':'. Hardcoding one is a +// mistake this repository has made before and it is not cosmetic: on Windows +// the list separator is ';' and ':' appears INSIDE every absolute path, so a +// ':'-keyed split cuts "C:\\x" into "C" and "\\x" -- the de-duplication then +// never matches and the joined value is a corrupt list. Caught by CI on +// Windows, not by any amount of reading. +std::vector> +resolve_env(const Info& info, const std::filesystem::path& subosDir) { + std::vector> out; + + const std::string subos = subosDir.string(); + auto expand = [&](std::string v) { + constexpr std::string_view kPh = "${subosdir}"; + for (auto pos = v.find(kPh); pos != std::string::npos; + pos = v.find(kPh, pos + subos.size())) + v.replace(pos, kPh.size(), subos); + return v; + }; + + const std::string sep = mcpp::platform::env::path_list_separator(); + + // Does the list already contain `value` as a WHOLE element? Compared + // element-wise rather than by substring: a plain `find` would consider + // "/a/bc" already present in "/a/bcd". + auto contains_element = [&](std::string_view list, std::string_view value) { + for (std::size_t i = 0; i <= list.size();) { + auto end = list.find(sep, i); + auto piece = list.substr(i, end == std::string_view::npos + ? std::string_view::npos : end - i); + if (piece == value) return true; + if (end == std::string_view::npos) break; + i = end + sep.size(); + } + return false; + }; + + // An explicit loop rather than std::ranges::find with a member-pointer + // projection into std::pair. The projection form reads well and crashed + // the clang 20.1.7 frontend outright when this module was compiled for + // the MSVC target -- a segfault with no diagnostic beyond "clang frontend + // command failed due to signal". Nothing here needs the fancier spelling. + for (auto const& p : info.providers) { + for (auto const& d : p.decls) { + auto value = expand(d.value); + std::pair* hit = nullptr; + for (auto& kv : out) + if (kv.first == d.var) { hit = &kv; break; } + if (!hit) { out.emplace_back(d.var, value); continue; } + if (d.op == "set") { hit->second = value; continue; } + if (!contains_element(hit->second, value)) + hit->second = value + sep + hit->second; + } + } + return out; +} + +} // namespace mcpp::xlings::subos diff --git a/tests/e2e/200_subos_env_reaches_program.sh b/tests/e2e/200_subos_env_reaches_program.sh new file mode 100755 index 00000000..81da7302 --- /dev/null +++ b/tests/e2e/200_subos_env_reaches_program.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env bash +# requires: elf +# 200_subos_env_reaches_program.sh — a subos's declared environment must reach +# the program mcpp launches (mcpp#352). +# +# A program needs three things: it links (bootstrap), it finds its libraries +# (RPATH), and it is told where its runtime data lives (env). mcpp supplied the +# first two and nothing for the third: xlings's graphics packages declare +# LIBGL_DRIVERS_PATH and friends into the subos, `xlings subos use` applied +# them, and `mcpp run` did not. That is why a GLFW binary could link cleanly +# and exit 255 with no output at all. +# +# The probe variable here is deliberately NOT a graphics one. mcpp does not +# know what any of these variables mean -- it carries whatever the subos +# declares -- and a test naming LIBGL_DRIVERS_PATH would quietly suggest +# otherwise. +set -euo pipefail + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +# A subos that declares one variable, in xlings's own schema. Built here rather +# than by mutating the developer's real subos: an earlier e2e wrote through a +# symlink and permanently broke a real toolchain, and MCPP_SUBOS_DIR exists so +# this test never has to go near one. +subos="$TMP/subos" +mkdir -p "$subos/usr/lib/dri" +cat > "$subos/.xlings.json" <<'EOF' +{ "workspace": {}, + "subos_info": { "schema_version": 1, "runtime": "glibc@2.39", + "envs": [ { "binding": "probe@1", "decls": [ + { "var": "MCPP_E2E_PROBE", "op": "prepend", + "value": "${subosdir}/usr/lib/dri" } ] } ] } } +EOF + +cd "$TMP" +"$MCPP" new hello > /dev/null +cd hello +cat > src/main.cpp <<'EOF' +#include +#include +int main() { + const char* v = std::getenv("MCPP_E2E_PROBE"); + std::printf("PROBE=%s\n", v ? v : "(unset)"); + return 0; +} +EOF + +# 1. `mcpp run` hands it to the program. +out=$(MCPP_SUBOS_DIR="$subos" "$MCPP" run 2>&1) || { + echo "mcpp run failed:"; echo "$out"; exit 1; } +echo "$out" | grep -q "PROBE=$subos/usr/lib/dri" || { + echo "the subos's declared environment did not reach the program:" + echo "$out" + exit 1 +} + +# 1b. THE SECOND RUN, which takes the cached fast path. +# +# This is the assertion that matters most, and the one a single-run test +# cannot make. The fast path builds its own child environment and skips +# prepare_build entirely; when it was first written it did not know about +# subos declarations at all, so a program worked on the run right after a +# build and silently stopped finding its runtime data on every run after +# that. For a GL application that is "it worked once and now the window is +# black", with nothing in between to attribute it to. +out_cached=$(MCPP_SUBOS_DIR="$subos" "$MCPP" run 2>&1) || { + echo "cached mcpp run failed:"; echo "$out_cached"; exit 1; } +# Self-check FIRST: prove this run actually took the fast path, otherwise the +# assertion below is vacuous and would keep passing after the coverage it +# exists for has silently gone away. The full path resolves the toolchain and +# says so; the fast path skips prepare_build entirely and never prints it. +echo "$out_cached" | grep -q 'Resolving toolchain' && { + echo "the second run did NOT take the cached fast path, so this test is" + echo " not covering it. Fix the test before trusting the assertion below:" + echo "$out_cached" + exit 1 +} +echo "$out_cached" | grep -q "PROBE=$subos/usr/lib/dri" || { + echo "the cached fast path dropped the subos environment — the program" + echo " gets a different environment on its second run than its first:" + echo "$out_cached" + exit 1 +} + +# 1c. ...and the environment is re-READ, not cached with the build. A user +# who installs a graphics stack between two runs must get it without +# rebuilding, so changing the declaration must change the next run. +sed -i 's#/usr/lib/dri#/usr/lib/dri2#' "$subos/.xlings.json" +out_changed=$(MCPP_SUBOS_DIR="$subos" "$MCPP" run 2>&1) || { + echo "run after changing the declaration failed:"; echo "$out_changed"; exit 1; } +echo "$out_changed" | grep -q "PROBE=$subos/usr/lib/dri2" || { + echo "the subos declaration changed but the program still sees the old" + echo " value — the environment was cached with the build instead of" + echo " being read from the subos:" + echo "$out_changed" + exit 1 +} +sed -i 's#/usr/lib/dri2#/usr/lib/dri#' "$subos/.xlings.json" + +# 2. Without a subos saying anything, nothing is invented. +out2=$("$MCPP" run 2>&1) || { echo "plain mcpp run failed:"; echo "$out2"; exit 1; } +echo "$out2" | grep -q 'PROBE=(unset)' || { + echo "a variable appeared with no subos declaring it:" + echo "$out2" + exit 1 +} + +# 3. A subos with no self-description degrades quietly and still runs. This is +# the state of every subos created before xlings grew the block, so it must +# not be an error. +bare="$TMP/bare" +mkdir -p "$bare" +echo '{ "workspace": {} }' > "$bare/.xlings.json" +out3=$(MCPP_SUBOS_DIR="$bare" "$MCPP" run 2>&1) || { + echo "a subos without subos_info broke the run:"; echo "$out3"; exit 1; } +echo "$out3" | grep -q 'PROBE=(unset)' || { + echo "unexpected output from a subos with no declarations:"; echo "$out3"; exit 1; } + +echo "PASS: subos environment reaches the program, and nothing is invented" diff --git a/tests/e2e/30_pack_modes.sh b/tests/e2e/30_pack_modes.sh index 0ef689b8..32f16b46 100755 --- a/tests/e2e/30_pack_modes.sh +++ b/tests/e2e/30_pack_modes.sh @@ -138,6 +138,28 @@ grep -q 'Hello' "$TMP/b-out.log" || { grep -q 'Hello' "$TMP/b-name.log" || { cat "$TMP/b-name.log"; echo "Mode B -entry output missing"; exit 1; } +# Launching through the loader makes the kernel set /proc/self/exe to the +# LOADER, so every "find my resources next to the executable" path silently +# resolves against lib/ instead. The wrapper must therefore hand the program +# an answer that survives — and it must hand the same one from both entry +# points, since they are documented as interchangeable. +for entry in run.sh myapp; do + grep -q 'MCPP_BUNDLE_DIR' \ + "$TMP/b/myapp-0.1.0-x86_64-linux-gnu-bundle-all/$entry" || { + echo "Mode B: $entry does not export MCPP_BUNDLE_DIR — an application" + echo " cannot locate its own resources under the bundled loader" + exit 1; } +done + +# The value must be the bundle ROOT (`$here`), not the loader's directory and +# not a path baked in at pack time — the bundle is relocatable, so anything +# absolute would be wrong the moment it is extracted somewhere else. +grep -q 'MCPP_BUNDLE_DIR="\$here"' \ + "$TMP/b/myapp-0.1.0-x86_64-linux-gnu-bundle-all/run.sh" || { + echo "Mode B: MCPP_BUNDLE_DIR is not derived from the bundle root (\$here)" + grep MCPP_BUNDLE_DIR "$TMP/b/myapp-0.1.0-x86_64-linux-gnu-bundle-all/run.sh" + exit 1; } + # ─── Mode `system` (Mode::None — depend on OS for all .so) ───────────── "$MCPP" pack --mode system > "$TMP/pack-sys.log" 2>&1 || { cat "$TMP/pack-sys.log"; echo "system pack failed"; exit 1; } diff --git a/tests/e2e/65_toolchain_runtime_dirs_for_run.sh b/tests/e2e/65_toolchain_runtime_dirs_for_run.sh index 1587ae2b..dad55814 100755 --- a/tests/e2e/65_toolchain_runtime_dirs_for_run.sh +++ b/tests/e2e/65_toolchain_runtime_dirs_for_run.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# requires: llvm linux +# requires: scan-deps elf # dlopen() providers such as GLX drivers do not use the main executable's # RUNPATH for their own DT_NEEDED closure. mcpp run must therefore expose the # toolchain runtime directories in LD_LIBRARY_PATH as well. diff --git a/tests/e2e/run_all.sh b/tests/e2e/run_all.sh index 5f10700e..392849c8 100755 --- a/tests/e2e/run_all.sh +++ b/tests/e2e/run_all.sh @@ -129,6 +129,40 @@ fi echo "Detected capabilities: ${CAPS[*]:-}" +# --------------------------------------------------------------------------- +# Every token a test may declare, across ALL platforms. +# --------------------------------------------------------------------------- +# A test declaring anything else is skipped on every runner, forever, and the +# skip line reads exactly like a legitimate one -- "missing capability: linux" +# is indistinguishable from "missing capability: msvc" on a Linux box. Two +# tests were in that state when this guard was added: 65_toolchain_runtime_ +# dirs_for_run.sh (`llvm linux`) had never run in CI at all, and it passes. +# +# This list is the UNIVERSE, not what this machine has: `msvc` is legitimately +# absent on Linux and must stay legal to declare. It is checked against the +# CAPS+=() calls above by tests/e2e/README or by reading them -- keep it in +# sync when adding a capability. +KNOWN_CAPS=(elf fresh-sandbox gcc import-std-libcxx macos mingw-cross msvc + musl nasm no-msvc pack patchelf scan-deps symlink unix-shell + windows wine) + +bad_tokens=0 +for tf in "$HERE"/[0-9]*.sh; do + base="$(basename "$tf")" + req="$(sed -n '2p' "$tf")" + [[ "$req" =~ ^#\ requires: ]] || continue + toks="${req#\# requires:}" + for tok in $toks; do + if [[ " ${KNOWN_CAPS[*]} " != *" $tok "* ]]; then + echo "ERROR: $base declares unknown capability '$tok' — it would be" + echo " skipped on every runner. Known: ${KNOWN_CAPS[*]}" + bad_tokens=1 + fi + done +done +[[ $bad_tokens -eq 0 ]] || exit 1 + + # --------------------------------------------------------------------------- # Helper: check if a test's requirements are satisfied # --------------------------------------------------------------------------- diff --git a/tests/unit/test_subos_info.cpp b/tests/unit/test_subos_info.cpp new file mode 100644 index 00000000..95385778 --- /dev/null +++ b/tests/unit/test_subos_info.cpp @@ -0,0 +1,180 @@ +// Reading the `subos_info` block xlings writes into a subos's .xlings.json. +// +// This is a CROSS-REPO CONTRACT, so the assertions are about the wire format +// and about what happens when it is absent or newer than we understand — +// not about internal behaviour. The degradation cases carry the weight: a +// subos made before xlings grew the block is the common case on any machine +// that has been around a while, and silence there is what made mcpp#352 +// expensive to find in the first place. + +#include + +import std; +import mcpp.platform; +import mcpp.xlings.subos_info; + +namespace su = mcpp::xlings::subos; + +namespace { + +struct Tmp { + std::filesystem::path dir; + Tmp() { + dir = std::filesystem::temp_directory_path() + / std::format("mcpp_subos_test_{}", std::random_device{}()); + std::filesystem::create_directories(dir); + } + ~Tmp() { std::error_code ec; std::filesystem::remove_all(dir, ec); } + void write(std::string_view body) const { + std::ofstream(dir / ".xlings.json") << body; + } +}; + +TEST(SubosInfo, ReadsRuntimeAndEnvDeclarations) { + Tmp t; + t.write(R"({ + "workspace": {}, + "subos_info": { + "schema_version": 1, + "runtime": "glibc@2.39", + "envs": [ + { "binding": "mesa@25.0.7.1", "decls": [ + { "var": "LIBGL_DRIVERS_PATH", "op": "prepend", + "value": "${subosdir}/usr/lib/dri" }, + { "var": "XDG_DATA_DIRS", "op": "prepend", + "value": "${subosdir}/share" } + ]} + ] + } + })"); + auto info = su::read(t.dir); + EXPECT_TRUE(info.present); + EXPECT_EQ(info.schema, 1); + EXPECT_EQ(info.runtime, "glibc@2.39"); + ASSERT_EQ(info.providers.size(), 1u); + EXPECT_EQ(info.providers[0].binding, "mesa@25.0.7.1"); + ASSERT_EQ(info.providers[0].decls.size(), 2u); + EXPECT_EQ(info.providers[0].decls[0].var, "LIBGL_DRIVERS_PATH"); + EXPECT_TRUE(info.note.empty()); +} + +// `${subosdir}` expands against the subos the block was read FROM, not +// against any global notion of "the" subos. That is what lets a program +// built under one subos run correctly under another. +TEST(SubosInfo, ResolvesSubosdirPlaceholder) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39", + "envs":[{"binding":"mesa@1","decls":[ + {"var":"LIBGL_DRIVERS_PATH","op":"prepend","value":"${subosdir}/usr/lib/dri"}]}]}})"); + auto env = su::resolve_env(su::read(t.dir), t.dir); + ASSERT_EQ(env.size(), 1u); + EXPECT_EQ(env[0].first, "LIBGL_DRIVERS_PATH"); + // Literal concatenation, NOT a path join. The separator in the declaration + // belongs to the subos manifest and is substituted verbatim; turning it + // into the host's would rewrite a value we do not own. On Windows the two + // spellings differ and the path-join form is the wrong expectation. + EXPECT_EQ(env[0].second, t.dir.string() + "/usr/lib/dri"); +} + +// Several providers may contribute to one variable — that is the normal +// shape for the graphics stack, where mesa and a vendor bridge both add an +// EGL vendor directory. `prepend` joins them; it must not drop either. +TEST(SubosInfo, PrependJoinsProvidersInOrder) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ + {"binding":"a-mesa@1","decls":[ + {"var":"V","op":"prepend","value":"${subosdir}/one"}]}, + {"binding":"b-vendor@1","decls":[ + {"var":"V","op":"prepend","value":"${subosdir}/two"}]}]}})"); + auto env = su::resolve_env(su::read(t.dir), t.dir); + ASSERT_EQ(env.size(), 1u); + const auto sep = mcpp::platform::env::path_list_separator(); + EXPECT_EQ(env[0].second, + t.dir.string() + "/two" + sep + t.dir.string() + "/one") + << "both providers must survive, later binding front-most"; +} + +// The same value arriving twice must not accumulate: nested invocations +// would otherwise grow the variable without bound. +TEST(SubosInfo, PrependDeduplicates) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ + {"binding":"a@1","decls":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}, + {"binding":"b@1","decls":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}]}})"); + auto env = su::resolve_env(su::read(t.dir), t.dir); + ASSERT_EQ(env.size(), 1u); + // One entry, not two. The de-duplication has to split on the PLATFORM's + // list separator: keyed on ':' it would cut "C:\\x" apart on Windows, + // match nothing, and grow the list on every nested invocation. + EXPECT_EQ(env[0].second, t.dir.string() + "/x"); + EXPECT_EQ(env[0].second.find(mcpp::platform::env::path_list_separator()), + std::string::npos); +} + +// `set` replaces rather than joins — xlings's own precedence. +TEST(SubosInfo, SetReplaces) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ + {"binding":"a@1","decls":[{"var":"V","op":"prepend","value":"/one"}]}, + {"binding":"b@1","decls":[{"var":"V","op":"set","value":"/two"}]}]}})"); + auto env = su::resolve_env(su::read(t.dir), t.dir); + ASSERT_EQ(env.size(), 1u); + EXPECT_EQ(env[0].second, "/two"); +} + +// A subos made before xlings grew the block. Degrade, and SAY SO — this is +// the common case on an existing machine, and mcpp's own sandbox subos was +// measured in exactly this state. +TEST(SubosInfo, MissingBlockDegradesWithANote) { + Tmp t; + t.write(R"({"workspace":{}})"); + auto info = su::read(t.dir); + EXPECT_FALSE(info.present); + EXPECT_TRUE(info.runtime.empty()); + EXPECT_FALSE(info.note.empty()); + EXPECT_TRUE(su::resolve_env(info, t.dir).empty()); +} + +// A schema newer than we understand: read what we can, and say we are behind. +// Refusing outright would make a newer xlings break an older mcpp, which is +// the failure mode the index-floor incident already paid for once. +TEST(SubosInfo, NewerSchemaIsUsedAndAnnounced) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":99,"runtime":"glibc@2.44","envs":[]}})"); + auto info = su::read(t.dir); + EXPECT_TRUE(info.present); + EXPECT_EQ(info.runtime, "glibc@2.44"); + EXPECT_FALSE(info.note.empty()); +} + +TEST(SubosInfo, NoFileAtAllIsNotACrash) { + Tmp t; // nothing written + auto info = su::read(t.dir); + EXPECT_FALSE(info.present); + EXPECT_FALSE(info.note.empty()); +} + +TEST(SubosInfo, MalformedJsonDegrades) { + Tmp t; + t.write("{ this is not json"); + auto info = su::read(t.dir); + EXPECT_FALSE(info.present); + EXPECT_FALSE(info.note.empty()); +} + +// The family mapping is part of the cross-repo contract: xlings derives it +// from the runtime string too, and the two must agree or a subos and a build +// will disagree about what ABI they are talking about. +TEST(SubosInfo, FamilyOfMirrorsXlings) { + EXPECT_EQ(su::family_of("glibc@2.39"), "linux-x86_64-glibc"); + EXPECT_EQ(su::family_of("musl@1.2.5"), "linux-x86_64-musl"); + EXPECT_EQ(su::family_of("glibc@2.39", "aarch64"), "linux-aarch64-glibc"); + EXPECT_EQ(su::family_of("wasi-libc@1"), "wasm32-wasi"); + EXPECT_EQ(su::family_of("macos_sdk@14.0", "arm64"), "darwin-arm64"); + EXPECT_EQ(su::family_of("ucrt@10"), "windows-x86_64-ucrt"); + EXPECT_EQ(su::family_of("nonsense@1"), "unknown"); + // No '@' at all is not a binding; it must not be read as one. + EXPECT_EQ(su::family_of("glibc"), "linux-x86_64-glibc"); +} + +} // namespace