From 1ef03f9fa72e0c5002d8b1b79cf979527aa32fdf Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 16:17:55 +0800 Subject: [PATCH 01/21] docs: 0.0.97 issue triage + architectural remediation implementation plan --- ...riage-215plus-architectural-remediation.md | 259 ++++++++++ ...ectural-remediation-implementation-plan.md | 453 ++++++++++++++++++ 2 files changed, 712 insertions(+) create mode 100644 .agents/docs/2026-07-18-issue-triage-215plus-architectural-remediation.md create mode 100644 .agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md diff --git a/.agents/docs/2026-07-18-issue-triage-215plus-architectural-remediation.md b/.agents/docs/2026-07-18-issue-triage-215plus-architectural-remediation.md new file mode 100644 index 00000000..5e11c1a0 --- /dev/null +++ b/.agents/docs/2026-07-18-issue-triage-215plus-architectural-remediation.md @@ -0,0 +1,259 @@ +# Issue #215+ 分类分析与架构级修复方案 + +> 日期:2026-07-18 +> 基线:mcpp **0.0.96**(HEAD `42198fb`,`MCPP_VERSION = "0.0.96"` @ `fingerprint.cppm:21`) +> 范围:GitHub issue #215 及其后全部 OPEN 项(#224/#225/#226/#227/#228/#229/#230/#232/#233/#234/#235),外加"`mcpp run` 构建过但运行前很慢"、"nasm 冷环境失败"、"mcpp.toml 相关"三条用户重点;并入 **R6**(index 仓测试模块包所需的默认命名空间重定向)与两条 workspace ergonomics 补充(root `[indices]` 一次声明全员可用、`mcpp run` 选择成员)。 +> 所有 file:line 锚点均在 0.0.96 源码上核实(五路并行 explorer 交叉验证)。 +> 上游语境:本批 issue 绝大多数来自 mcpplibs 把 **ffmpeg-m / opencv-m 以"全源码直编"形态**收录时的踩坑(参见 [2026-07-17-asm-sources-and-general-build-capabilities-design.md](2026-07-17-asm-sources-and-general-build-capabilities-design.md) 的下一波),是 0.0.95 声明式清单能力落地后暴露的第二层缺口。 + +--- + +## 0. 结论先行 + +**先纠正一个直觉**:用户提示"mcpp.toml 相关的问题可能是一类"——实测**不是一类**。它们分裂成两个不同根因:命令行**装配语义**(#226/#234,把 flag 当不透明串)与解析器**语法表达力**(#227/#228,parser 缺 array-of-tables / glob 缺花括号)。而 `mcpp run` 慢(#225)与 nasm(#232)各自独立成类。#230 **已在 0.0.96 修复**(根因是 scanner 崩溃,非 nasm/子进程,详见 §7)。 + +按**根因所在子系统**归并,收敛为 **6 个可独立成 PR 的簇 + 2 个独立项**: + +| 簇 | issue | 根因一句话 | 触及子系统 | 量级 | 建议 PR | +|---|---|---|---|---|---| +| **A** flag 装配模型 | #226 #234 | flag 是不透明 `vector`:include 重写只认 `-I`、含空格值不转义 | scanner/flags/ninja_backend | 中 | **PR-1** | +| **B** 清单语法表达力 | #227 #228 | 自研 TOML parser 全局拒 `[[table]]`;glob 匹配器无 `{a,b}` | libs/toml + scanner glob | 中 | **PR-2** | +| **C** 构建图节点身份与输入完备 | #233 #235 | 对象路径按"父目录名+文件名"折叠必撞;扫描产出的 `.dep` 被丢弃、编译边根本无 depfile | plan/ninja_backend/dyndep | 中 | **PR-3** | +| **D** 条件源集统一求值 | #229(=#218 复发) | cfg 条件 sources 只在 root/version-dep 求值,path/git dep 漏;feature sources 已统一而 cfg 没有 | prepare 条件求值漏斗 | 小-中 | **PR-4** | +| **E** 源发现范围 + run/build 缓存复用 | #225(#228 glob 顺带) | glob 永远从 root 全树遍历、只排 `.mcpp`;`mcpp run` 每次重扫、build 缓存不存源清单/产物路径 | scanner/execute | 中 | **PR-5** | +| **F** 外部工具供给统一门 | #232(#230 已修) | nasm 走 bespoke 弱机制(无索引刷新前置、失败降级为 warning、guard 吞真错),未接入工具链那条同步供给门 | prepare/xlings/fetcher | 中 | **PR-6** | +| **G** workspace 配置与测试基建 | #224 + **R6** + 2 补充 | 继承只传 `version` 丢 `path`/根锚点;相对 `[indices]` 按成员目录解析;默认命名空间**结构上不可重定向**;`mcpp run` 无成员选择 | project/manifest/prepare/cli | 中 | **PR-7** | +| 不做 | #215 | 上游 Clang 落地 P2996 后才加一行,现无从动手 | cppfly | 微 | **不做** | + +**版本建议**:A/B/C/D/E 是 ffmpeg-m/opencv-m 全源码直编的硬阻塞,建议合成一个版本里程碑(**0.0.97**)分 5 个 PR 顺序合入;F 可并行;#224 独立;#230 只需在 windows CI 用 0.0.96 pin 复验后关闭;#215 挂 upstream。 + +**贯穿全文的一条架构主线**:mcpp 目前多处把"结构化意图"过早压平成字符串/单点特判,再在下游试图恢复——A(flag 边界丢失)、C(对象名折叠丢路径)、D(条件求值散在多个调用点)、E(glob 前缀信息被丢弃后靠遍历补)、F(工具供给不复用统一门)都是同一病理的不同切面。每个簇的"架构方案"都指向**把意图保留到唯一的收敛点**,而非再加一处特判。 + +--- + +## 1. 分类方法与"一类"的判据 + +判定"能否放进一个 PR"的标准不是"issue 表面相似",而是**三个必须同时成立**: + +1. **同根因子系统**——改动落在同一组文件/同一个抽象层,一个 reviewer 能一次性把握; +2. **同回归面**——共享同一批测试夹具与验证路径(避免一个 PR 拆两拨 e2e); +3. **无语义耦合冲突**——两处改动不互相改写对方的中间表示(否则应先做基础层)。 + +据此,用户直觉里的"mcpp.toml 一类"被拆开:#226/#234 落在**命令行装配**(scanner→ninja_backend 的 flag 流),#227/#228 落在**解析层**(libs/toml + glob 匹配器)——两层之间只有数据流经过、无共享逻辑,强行同 PR 会让 diff 横跨"parser 改造 + 命令装配改造"两个不相干高风险区。反过来,#233/#235 表面一个是"路径撞名"一个是"改 .inc 不重编",却**同根**:都在 `ninja_backend.cppm` 的构建边生成、都因"扫描已拿到的信息没并进 ninja 依赖边"而坏——故合一簇。 + +--- + +## 2. 簇 A:flag 装配模型(#226 + #234)—— PR-1 + +### 2.1 现象 +- **#226**:`[build].cflags/cxxflags` 里相对 `-Ihdr` 会被绝对化(命令行可见),但 `-iquotehdr` / `-isystem` / `-idirafter` **不重写**,按 ninja 构建目录解析 → 头文件静默找不到。 +- **#234**:`defines = ["T=long long"]` 未加引号拼进命令行,g++ 把 `long` 当独立输入文件 → `linker input file not found`。 + +### 2.2 根因(核实) +flag 在整条管线里是**不透明预拼字符串**,没有任何 `Flag{kind,value}` 结构: +- 数据模型:`GlobFlags`(`types.cppm:125-131`)、`BuildConfig.cflags/cxxflags/ldflags`(`types.cppm:137-156`)、`SourceUnit.packageC*flags`(`graph.cppm:15-29`)全是 `vector`;全局 flag 更早在 `flags.cppm:312-331` 就压平成单个 `std::string` blob。 +- **有损融合点**:`apply_glob_flags`(`scanner.cppm:571-586`)把 define 拼成 `"-D"+d`——`-DT=long long` 在此刻成为**一个含内部空格的 vector 元素**,参数边界意图就此丢失。 +- **include 重写只认 `-I`**:`absolutize_include_flags`(`scanner.cppm:338-350`)用字面 `f.starts_with("-I")`(342)+ 硬编码 `substr(2)`(343)+ 硬编码 `"-I"` 重建(347)。`-iquote/-isystem/-idirafter` 首字母是小写 `-i`、且有 joined/separated 两种拼写,根本不进这个判断。注意 `-I` 本已被 `Dialect::includePrefix`(`dialect.cppm:27,74`)抽象,这里却绕过 dialect 硬编码。 +- **发射端零转义**:`join_flags`(`ninja_backend.cppm:99-106`)就是 `out += ' '; out += flag;`。现有 `escape_ninja_path`/`escape_flag_path`(`ninja_backend.cppm:60-88`、`flags.cppm:80-90`)只做 ninja **语法**转义且只用于**路径**,不解决 shell 参数边界。 + +一句话:**边界与角色信息在 `scanner.cppm:571-586` 被抹掉,到 `ninja_backend.cppm:99` 想转义也已无从分辨** `-DT=long long`(一个参数须整体加引号)与 `-isystem /a/b`(两个 vector 元素、本就是两参数)。 + +### 2.3 架构方案(非临时) +**引入结构化参数中间表示,把"边界 + 角色"保留到发射点。** + +- 定义 `struct Arg { enum Kind { Plain, IncludeDir, Define, Separated } kind; std::string spelling; std::string value; };` 作为 per-glob/per-unit flag 的**规范内部形式**(替换裸 `vector` 的语义,发射时才铺开成命令行 token)。 +- **规范化 pass**(唯一收敛点,放在 `apply_glob_flags`/scanner 附加处): + - include 族——识别 `{-I, -iquote, -isystem, -idirafter, -iprefix, -L}` 全家族(joined 与 separated 两拼写),路径部分统一相对项目根绝对化,拼写走 `Dialect` 而非硬编码。把 `flags.cppm:161-165` 那条**另一路** includeDirs 绝对化也并进来,消除两套逻辑。 + - define——`value` 原样保留(含空格),不在此处拼 `-D`+串。 +- **发射 pass**(`join_flags` + `flags.cppm:312-331` blob 装配两个 choke point):按 `Arg.kind` 生成 token,含空格/特殊字符的 `value` 施加 **shell 引号 + ninja `$` 转义**双层(叠在现有 `escape_*` 之上)。 +- **一致性单测**:枚举 `-I/-iquote/-isystem/-idirafter` × 相对/绝对 × cflags/cxxflags/asmflags 三通道,断言绝对化;define 值含空格断言命令行单参数完整。 + +> 落地边界:本 PR **只碰 flag 流**(scanner 附加 → CompileUnit → ninja 变量),不动 sources/依赖解析。`Arg` 可先以最小形态落地(保留"是否 include 族 + 是否需整体引用"两个 bit),不必一步到位做全类型系统——关键是**边界不再在中途丢失**。 + +### 2.4 与 0.0.95 的历史呼应 +asm 设计文档 §1.6 已把 #226 记为 G8b("相对 `-I` 对 `.cppm` 不生效,先加复现再修,随 flags PR 顺手"),此 PR 即其正式兑现,并把范围从"补一个漏"升级为"根治不透明串"。 + +--- + +## 3. 簇 B:清单语法表达力(#227 + #228)—— PR-2 + +### 3.1 现象 +- **#227**:`[[build.flags]]` 标准 TOML 数组表被拒(`array-of-tables not supported`),长条目(NASM 8 个 `-I` + `-Pconfig.asm`,单行 300+ 字符)只能挤一行。 +- **#228**:glob 不支持 `{a,b}`,`libavcodec/{aac,bsf,hevc,opus,vvc}/**` 触发 matched-no-source 警告,只能写 5 条重复条目。 + +### 3.2 根因(核实) +- #227 的拒绝在**词法/解析库层、全局**:`libs/toml.cppm:474-476` 见 `[` 直接 `return unexpected("array-of-tables not supported")`,注释("mcpp doesn't use them")说明是刻意未实现。mcpp.toml 走这个 parser(`toml.cppm:7` import),故 `[[build.flags]]` 在 tokenize 期就死,`toml.cppm:692` 的消费分支根本到不了。xpkg/Lua 路径反而原生支持有序数组表(`xpkg.cppm:864-919`)——两条 ingest 表达力不对等。 +- #228:全仓唯一 glob 匹配器 `path_matches_glob`(`scanner.cppm:103-163`)的递归 `match`(126-161)只支持 `**`/`*`/字面量,`{` 当普通字符,故 brace glob 永不命中。 + +### 3.3 架构方案(非临时) +两处都是**"closed syntax, open vocabulary"下的纯语法补齐,零新语义**(符合 schema 准入规则): + +- **#227**:教 `libs/toml.cppm` 的解析主循环(460-499,`[` 分支 471)把 `[[table]]` 累积成一个数组 `Value`;再扩 `toml.cppm:692-730` 让 `build.flags` **同时接受**内联表数组与数组表两形式(声明顺序=应用顺序语义不变)。这是补一个**通用 TOML 能力**,顺带让未来任何配置段都能用数组表。 +- **#228**:在 `expand_glob` **入口处**把 `{a,b}` 脱糖成多个 glob(而非改 `match` 递归)——理由:脱糖与簇 E 的"前缀收窄遍历"天然复合(先脱糖再对每个分支取前缀),且不污染热路径匹配器。备选:flags 条目的 `glob` 键接受数组(更小、解析层),二者可都做。 + +> 落地边界:纯 parser + glob 入口,**不碰**命令装配与构建图。可与 PR-1 并行(无文件重叠除 glob 入口,而 PR-1 不碰 glob)。 + +--- + +## 4. 簇 C:构建图节点身份与输入完备(#233 + #235)—— PR-3 + +### 4.1 现象 +- **#233**:`a/src/util.cpp` 与 `b/src/util.cpp` 都折叠成 `obj/repro_src/util.cpp.ddi` → `ninja: multiple rules generate`。是 OpenCV/LLVM 式 `modules//src/*.cpp` 标准布局的直接杀手(opencv-m 现被迫生成 457 个转发 stub)。 +- **#235**:模块 purview 内 `#include "vals.inc"` 改动后 `mcpp build` 判 up-to-date(0.01s),产物 stale。ffmpeg-m(978 函数)/opencv-m(~2000 名字)的 `gen_exports/*.inc` 导出面全走这个模式。 + +### 4.2 根因(核实) +- #233:对象路径唯一赋值点 `plan.cppm:450-462`。折叠逻辑 `455`:`parentDir = u.path.parent_path().filename()` ——只取**直接父目录名**,丢掉相对路径其余部分;`456-459` 拼成 `obj/_/`。`basenameCount`(430-433)只检测 basename 撞名,而它的"解法"(父目录名前缀)**本身不唯一**、且无最终唯一性断言。`.ddi` 路径(`ninja_backend.cppm:620`)从 `cu.object.parent_path()` 派生,故连带撞。 +- #235:**信息本已产出却被丢弃**。P1689 扫描命令带 `-M -MM -MF $out.dep`(`ninja_backend.cppm:531`)确实生成了含全部文本 include(purview + GMF + 普通头)的 depfile,但:① `cxx_scan` 规则**没有 `depfile=`/`deps=gcc`**(对比 nasm 规则 `:463-468` 是有的),`$out.dep` 生成即弃;② 计划期扫描 `p1689.cppm:350-365` 同样带 `-MF {dep}` 却只解析 `.ddi`,`.dep` 从不读;③ 更根本——`cxx_module`/`cxx_object` 编译规则在**非 MSVC 下根本没有任何 depfile**(`ninja_backend.cppm:402-435`,只有 `msvcDeps` 分支加 `deps=msvc`)。即**扫描的 `.dep` 是唯一曾捕获过 include 的地方,而它被丢在地上**;到达 ninja 的模块依赖边只有 `逻辑名→BMI`。 + +### 4.3 架构方案(非临时) +- **#233:对象路径镜像源相对路径。** 唯一改点 `plan.cppm:450-462`:把前缀从 `parent_path().filename()` 改为**源文件相对(包/项目根)完整路径**(sanitize 分隔符),即 `obj//a/src/util.cpp.o`;`.ddi/.dd/compile_target` 全部从 `cu.object` 派生故自动跟随。**追加一条 post-loop 断言**:所有 `cu.object` 互异(把"撞名"从运行期 ninja 错误提前成 mcpp 层可诊断错误)。基名撞名启发式(430-433)可退役。 +- **#235:给编译边加真正的 depfile 追踪。** 首选**方案 2**(而非只捞扫描 `.dep`):给 `cxx_module`/`cxx_object` 加 `-MD -MF $out.d` + `depfile = $out.d` + `deps = gcc`(镜像 nasm 规则 `:463-468`)。这在**编译期**捕获所有头/purview/GMF include,**一次性根治整类 stale**——不止 #235 的 purview,还包括至今非 MSVC 下**普通头文件改动也不重编**的潜伏 bug(本次探查的额外发现)。若要更省(不重复编译期扫描),备选方案 1:让 `mcpp dyndep` 收集器(`dyndep.cppm:314-340`)读每个 `.ddi` 旁的 `.dep`、把文件列表作为额外 implicit input 并进 dyndep 边——但这只覆盖模块 TU,不如方案 2 普适。 + +> 落地边界:两改都在 `plan.cppm` + `ninja_backend.cppm` 构建图生成,同组文件同 reviewer。回归夹具:#233 用双 `src/util.cpp`,#235 用 purview `#include .inc` 改值后断言重编;顺带加"改普通头触发重编"用例锁住方案 2 的额外收益。 + +--- + +## 5. 簇 D:条件源集统一求值(#229,= #218 复发)—— PR-4 + +### 5.1 现象 +`[target.'cfg(...)'.build].sources`(0.0.95 #223 引入)在包**自身 build** 时正常,但该包**作为 path 依赖被消费**时不展开 → undefined reference。与 0.0.94 修的 #218("feature sources 在 `mcpp test` 下不编译")**同类**:条件源集只在主构建路径求值,次级路径缺失。 + +### 5.2 根因(核实)—— 这是一个"漏斗未统一"的架构缺陷 +- 存储:`ConditionalConfig`(`types.cppm:252-268`,含 `sources`)、`Manifest.conditionalConfigs`(`types.cppm:367`);parser(`toml.cppm:827-862`、`xpkg.cppm:930-986`)只**存**不并入 base。 +- 求值/合并漏斗:`merge_conditional_build`(`prepare.cppm:345-362`)对命中谓词把 `cc.sources` APPEND 进 `buildConfig.sources`(358)与 `modules.sources`(359,scanner 实际走这个)。 +- **关键不对称**:该漏斗只在 **root**(`prepare.cppm:792`)与 **version/registry dep**(`prepare.cppm:1714`,#223 加)被调用;**path/git dep 走另一条分支**(`prepare.cppm:2447-2459` 裸 `manifest::load` → 直接 `makePackageRoot`),**从不调用** `merge_conditional_build` → path dep 的 cfg sources 存了却永不并入 → scanner 看不到 → undefined reference。**这就是 #229**。 +- 对照 feature sources:`apply` lambda(`prepare.cppm:2564-2668`)的 drop+add,#218 后 ADD 被移出 `!includeDevDeps` 门,且 `apply` **已对每个包**在每种模式调用(root `2689` + 每个 dep `2710`)——feature 维度**已统一**,cfg 维度**没有**。 + +### 5.3 架构方案(非临时)—— 建立"每包有效源集"唯一漏斗 +把 cfg 源求值折进 feature sources 那个**已经存在的 per-package 循环**(`prepare.cppm:2669-2711`),使**每个包(packages[0..n])在每种模式(build/test/workspace)走同一段"解析有效源集 = base + cfg + feature、按已解析 target 求值"**——正是 #218 统一 feature sources 的同款做法。具体: +1. 在 per-package 循环内、modgraph 扫描前,对 `pkg.manifest` 调用 `merge_conditional_build`(此处 manifest 已可原地改,且在 feature 激活后、扫描前的正确时点); +2. **退役**现有两个散点调用(root `792` 的 sources/flags 半、version-dep `1714`)以免双重合并; +3. **保留** root-only 的条件*依赖*合并(`800-802`,必须在依赖解析前),这是合法的 root 专属。 + +> 这条修复的价值不止 #229:它把"条件源集必须在所有构建路径统一求值"从**惯例**(#218 靠 code review 记住"add 要在所有模式做")升级成**结构不变量**(只有一个漏斗,想漏都难)。任何未来新增的条件源维度自动被覆盖。 +> 落地边界:纯 `prepare.cppm` 漏斗重排,回归夹具用 #229 的 `cfg(linux) sources` path dep + `mcpp test` 双路径(呼应记忆里"诊断 feature 必须 build/test 双路径对比")。 + +--- + +## 6. 簇 E:源发现范围 + run/build 缓存复用(#225)—— PR-5 + +### 6.1 现象 +`mcpp run --version` 每次 real≈9.5s(其中 build 只 0.03s),strace 见 ~143 万次文件操作,几乎全在无关子模块(`compat/node` 37 万、`compat/bun` 13 万路径引用)。同一二进制直接跑 0.00s,`mcpp build` 缓存命中 0.04s。即**延迟全在 run 的解析/发现路径**。 + +### 6.2 根因(核实)—— 三个叠加缺陷 +1. **无界遍历**:`expand_glob`(`scanner.cppm:233-287`)**永远从 `root` 起** `recursive_directory_iterator`(251-252),glob 只在**遍历后**逐文件 `path_matches_glob`(276)做词法过滤。`src/**/*` 仍访问整棵树。**从不从 glob 的固定前缀 `src/` 起走。** +2. **排除面太窄**:df985df 只加了 `.mcpp` 剪枝(`scanner.cppm:261-264`);**无** `.git`/git 子模块/`target`/vendored 树排除;且仍 `follow_directory_symlink`(252),只有 canonical 环检测。 +3. **run 每次重扫、缓存复用不了**:`mcpp run` → `build_run_target`(`execute.cppm:387-431`)**无条件** `prepare_build(false)`(391),不像 `cmd_build` 有 `try_fast_build` 快路径(`cmd_build.cppm:81-90`)。原因:run 事后要用 `ctx->plan.linkUnits` 定位二进制(397-410),而 plan 只是 `prepare_build` 的产物。而 `target/.build_cache`(`BuildCacheEntry`,`execute.cppm:34-41`)**只存**够重跑 ninja 的字段,**不存解析后的源清单、不存二进制目标路径**——run 无可复用。 + - 附带:`try_fast_build` 自己还有**第二份**硬编码到 `src/` 的独立遍历做新鲜度检查(`execute.cppm:317-328`),非 glob 感知。 + +### 6.3 架构方案(非临时) +唯一 choke point 是 `expand_glob`/`expand_dir_glob`,所有源发现都过它: +1. **前缀收窄遍历**:从 glob 提取首个通配前的字面前缀(`src/`),`recursive_directory_iterator` 从 `root/prefix` 起,而非 `root`。单点改,全体调用者受益。与簇 B 的 `{a,b}` 脱糖复合(脱糖后每分支各取前缀)。 +2. **边界排除**:在 `.mcpp` 剪枝旁,对 `.git`、`target`、`.gitmodules` 登记的子模块路径 `disable_recursion_pending()`;子模块边界默认排除除非显式声明为 member/package。 +3. **共享解析缓存,让 run 跳过重扫**:扩 `.build_cache` schema 持久化**解析后的二进制目标输出路径**(+ 已有 runtime env),以既有 fingerprint(`prepare.cppm:2936`)为有效性键;给 `build_run_target` 加与 `cmd_build` 平行的快路径:缓存有效则直接定位并启动 exe,不 `prepare_build`。顺手把 `try_fast_build` 那第二份 `src/` 硬编码遍历统一到前缀收窄的 `expand_glob`。 + +> 落地边界:`scanner.cppm`(遍历)+ `execute.cppm`(缓存 schema/快路径)。回归夹具:含大无关目录/子模块的工程,断言 `mcpp run` 与缓存 `mcpp build` 耗时同量级(#225 明确要求)。这是"运行前很慢"的正解——**不是**加个跳过开关,而是让发现有界 + 让 run 复用 build 已解析的结果。 + +--- + +## 7. 簇 F:外部工具供给统一门(#232;#230 已修)—— PR-6 + +### 7.1 #232 现象 +含 `.asm` 的包冷环境构建:`error: NASM sources present but no usable nasm ...` **先于** `Bootstrap nasm into mcpp sandbox (one-time)` 打出(~0.25s 后),构建已判死;两轮独立 run 时序一致;PATH 有 nasm 则不受影响。 + +### 7.2 #232 根因(核实)—— nasm 是一条**未接入统一供给门的弱机制** +- nasm 消费边 `prepare.cppm:3019-3032`: + ``` + auto cfgNasm = get_cfg(); // 默认 requireBootstrap=true + if (cfgNasm) nasmBin = xlings::ensure_nasm(...); // ← guard + if (!nasmBin) return unexpected("...no usable nasm..."); + ``` +- `ensure_nasm`(`xlings.cppm:1276-1295`)链:PATH(`which`+版本)→ sandbox(`find_sandbox_nasm`)→ 打印 "Bootstrap"(1287)→ `install_with_progress`(1289,**同步** join `xlings.cppm:993`)→ 再 `find_sandbox_nasm`。**它技术上是同步的**,但两个架构缺陷制造了报错先行: + 1. **guard 吞真错**:冷环境 `get_cfg()` 的 `check_base_init` 失败(`prepare.cppm:636-644`)返回 error-expected,`if(cfgNasm)` 为假 → `ensure_nasm` **根本没被调**(故无 "Bootstrap" 行)→ 直接抛 "no nasm";而 `get_cfg()` 的**真实 bootstrap 错误被丢弃**。 + 2. **无索引刷新前置**:`ensure_nasm` 装前不刷新 xlings 包索引(对比 fetcher 路径 `package_fetcher.cppm:745-746` 会 `ensure_official_package_index_fresh`),冷环境 `xim:nasm` 尚不在索引 → 装失败 → **仅 warning**(1290-1293)→ `find_sandbox_nasm` 返 nullopt → 抛错。失败根因再次被降级丢弃。 + +对照**正确范式**——工具链自动装是**同步 provision-before-edge 门**:`Fetcher::resolve_xpkg_path(target, autoInstall=true, &progress)`(`prepare.cppm:851/958-959/1080`,impl `package_fetcher.cppm:644-762`),它**先刷索引**(745-746)→ 阻塞 `install`(753)→ **校验 payload 落地**(758-762)→ 有 XLINGS_HOME 传播兜底(764-779);失败是**硬错**不是 warning。 + +### 7.3 #232 架构方案(非临时)—— nasm 走同一道门 +把 `prepare.cppm:3019-3032` 的 bespoke `ensure_nasm` + `if(cfgNasm)` guard **替换为工具链同款** `resolve_xpkg_path("xim:nasm", autoInstall=true, ...)`(或抽一个 `ensure_tool(pkg)` 共享 helper,工具链与 nasm 都调),使 nasm 继承:索引刷新前置、失败硬错、payload 校验、copy-from-global 兜底。并**修 guard**:不再静默丢 `get_cfg()` 的 bootstrap 错、config 加载失败时不跳过供给。`xlings::ensure_nasm` + `install_with_progress` 可退役/吸收。 + +> 更广的收益:任何未来"惰性供给的构建期工具"(未来的 protoc/bison/flex 之类)都应走这道门——**工具供给不该有第二套弱机制**。这与记忆里 mingw-cross/msvc 供给都收敛到 registry/fetcher 的方向一致。 + +### 7.4 #230:**已在 0.0.96 修复**(不入本 PR) +探查推翻了 issue 的初步怀疑(非 #220 nasm / #222 build.mcpp 子进程)。真相:windows "exit 127" 实为 `0xC0000409`(`__fastfail`,git-bash 报成裸 127),源自 module scanner 未捕获的 `std::system_error`——glob 遍历经 `.mcpp` 符号链接逃逸进 index checkout,对 CJK 文件名做窄串转换时抛异常。**正是 `df985df`("prune .mcpp from glob walks; never throw on unnarrowable names (#230)")修的**——commit 消息直接引用 #230。修复在 `scanner.cppm`(~110-124 narrow guard + 261-264 剪枝)+ `main.cpp:7-18`(last-resort catch,返 70 不 fastfail)。**行动**:mcpp-index windows CI 用 0.0.96 pin 复验后关闭 #230,无需代码。 +> 附带硬化(可选、低优):`build_program.cppm:499` 的 `build.mcpp.bin` **缺 `exe_suffix`**(nasm/其他工具都用 `platform::exe_suffix`),windows 上是潜伏隐患;`prepare.cppm:2395-2415` 的 git clone 是仅存的裸命令名 spawn(无 `which` 前置)。二者可随 F 顺手,但非 #232/#230 的成因。 + +--- + +## 8. 簇 G:workspace 配置与测试基建(#224 + R6 + 2 补充)—— PR-7 + +这四项**同根因子系统**(workspace 配置继承 + `[indices]` 命名空间路由),且**同一受益方**:让 index 仓能用一条 `mcpp test --workspace` 声明式地验证所有包(含模块包),删掉一整套 shell 伪造补丁。故合一簇一 PR。 + +### 8.1 病理总纲:workspace 的"根锚点"与"命名空间路由"都不完整 +mcpp workspace 有 members 枚举与逐成员构建,但两个横切能力有洞: +1. **继承的相对配置无根锚点**——继承下来的 `path`/`[indices]` 相对值按**消费成员目录**而非 **workspace 根**求值; +2. **命名空间路由无"默认"入口**——`[indices]` 只能重定向**有名字**的命名空间,默认命名空间(`namespace = ""`)在两处被**硬编码短路**到 builtin index,`m->indices` 根本不被查。 + +这两个洞叠加,逼出 index 仓的 shell 补丁(R6)与逐成员重复声明(#224)。 + +### 8.2 #224:继承只传 version + 相对路径按成员目录重基("根添加一次不 ok") +- **根因**: + - 复现 1(path dep 走 `workspace.dependencies`):`merge_workspace_deps`(`project.cppm:52-75`)只把 `spec.version` 从 workspace 抄给成员、**丢弃 `path`**,于是本地 path 依赖被当 index/semver 解析报 "isn't cloned locally yet"。 + - 复现 2(root `[indices]` 重基):workspace 索引**仅当成员自己没声明时**才继承(`prepare.cppm:577-579`、`600-602`,`if (m->indices.empty())`),且相对 `path` 值(如 `mcpp`)按**成员目录**解析成 `//mcpp` 而非 `/mcpp`;嵌套成员深度还改变含义(须 `../mcpp`、`../../mcpp`)。**这正是用户说的"根添加一次就 ok 不成立"**——每个成员被迫重复声明各自相对路径。 +- **架构方向**:解析后的 workspace 配置**保留 workspace 根目录**为锚点;所有**继承而来**的相对路径(`[indices]`、path dep)一律相对**根**求值(而非消费成员);`merge_workspace_deps` 传播 `path` 时连带记录"相对根"来源。 + +### 8.3 R6:默认命名空间无法重定向(核心新增)—— 补一个语法空洞 +- **背景**:index 仓本身是个 workspace,每个包配测试工程,用 `[indices] compat = { path = "../../.." }` 把 `compat` 命名空间指向本仓 checkout,PR 合并前即可 `mcpp test --workspace` 真实验证描述符。**但** imgui/ffmpeg/opencv 三个公开**模块包走默认命名空间**(描述符 `namespace = ""`,用户写 `[dependencies] opencv = "0.0.2"` 不带前缀),而 `[indices]` **没有语法能把"裸的默认命名空间"指到本地路径**。 +- **现状替代品(要删的补丁)**:三个 ~100 行 `tests/smoke_*_module.sh`——建临时 `MCPP_HOME` → 把本仓 checkout 整个拷贝覆盖到默认 index 的落盘位置(`registry/data/mcpplibs`)→ 造临时消费工程 `mcpp build && mcpp run`;CI 还得为它们各开一个专属 job。**能用但丑**:三份近似复制的 shell(违背 index 仓 "zero-shell 自举" 立仓哲学,cf. [记忆:workspace-test-and-zero-shell-index])、易碎的 MCPP_HOME 重播/缓存链接/nasm 沙箱索引刷新细节、CI job 数随模块包线性增长。这个空洞在 imgui 时代就被 index 仓注释标记"待架构重构解决"。 +- **根因(核实)—— 默认命名空间被两处硬编码短路,`[indices]` 从不被查**: + - `usesBuiltinIndex`(`prepare.cppm:1134-1145`):default-ns dep 在 `1140` `if (ns == kDefaultNamespace) return true;` **直接短路**,`m->indices.find(ns)`(1142)对默认命名空间永不执行; + - `findIndexForNs`(`prepare.cppm:1297-1310`):`1300` `if (ns.empty() || ns == kDefaultNamespace) return nullptr;`(nullptr = builtin)**又一处短路**,同样在查 `m->indices` 之前。 + - 即:默认命名空间→builtin index 是**结构写死**的,`[indices]` 只对有名字的命名空间生效。`[indices]` 解析(`toml.cppm:924` 起)本就迭代任意键,空引号键 `"" = {...}` TOML 可表达——**缺的不是解析,是路由未查表**。 +- **架构方案(非临时)**:让默认命名空间可被 `[indices]` 重定向。定一个**规范拼写**映射到 `kDefaultNamespace`(建议保留 token `default = { path = ... }`,比空引号键 `""` 更可读、更不易误写;二者可都接受);两处短路(`1140`、`1300`)改为**先查** `m->indices` 是否有默认命名空间的重定向条目,有则用之、无则回落 builtin。这是**把"默认命名空间不可寻址"这个语法空洞补上**,不是给用户加功能——补完后三个模块包各建一个和 `opencv5` 一样的普通测试工程(只多一行 `default = { path = "../../.." }`)加入 workspace members:**三个 smoke shell 删除、三个专属 CI job 删除、模块包验证并入 `mcpp test --workspace` 一条命令**,以后每加模块包只加一个成员目录。 + +### 8.4 补充:`mcpp run` 不支持选择 workspace 成员 +- **根因(核实)**:`cmd_run`(`cmd_build.cppm:99-107`)只读一个位置参数(二进制目标名)+ passthrough,**不解析 `--package`、不做 workspace fan-out**;对比 `cmd_build`(读 `--package` `cmd_build.cppm:47`)、`cmd_test`(读 `--package` `:121` + `workspace_fanout_members`)。`resolve_member_dir`(`project.cppm:87-106`)这条 build/test 已共享的成员选择规则,run 完全没接。 +- **架构方案**:`cmd_run` 接入与 build/test **同一** `--package` 解析 + `resolve_member_dir`,`build_run_target` 增加 `package_filter` 形参、在成员目录 scope 内解析并启动该成员的二进制。语义与 `mcpp build -p ` / `mcpp test -p ` 对齐(run 天然单成员,不需要 `--workspace` fan-out)。 + +### 8.5 PR-7 边界与顺序 +- 触及 `project.cppm`(继承/根锚点)、`prepare.cppm`(两处命名空间短路 + 索引继承重基)、`manifest/toml.cppm`(`default` 键规范化)、`cli/cmd_build.cppm`(run 成员选择)。与 A–F **无文件重叠**(F 也碰 prepare 但在 nasm 供给段 3019-3032,与此处 577/1134/1297 不冲突),可完全并行。 +- 回归夹具:虚拟 + rooted workspace、不同深度成员、root `[indices]` 相对路径一次声明全员可用、`default = {path}` 重定向默认命名空间命中本地包、`mcpp run -p `。**收尾验证**:在 mcpp-index 侧把一个模块包改成 workspace member 测试工程、删对应 smoke shell + CI job,`mcpp test --workspace` 全绿——这是 R6 真正落地的判据。 + +## 9-pre. #215:cppfly Clang 反射行 —— 不做 +上游 Clang(非 bloomberg fork)落地 P2996 后才谈得上给 `cppfly.cppm` 的 `kReflectionRules` 加一行(且版本号 + 旗标拼写须实机 `-fsyntax-only` 特性宏核实,不得臆测)。截至 2026-07 上游无。**标记为不做**:不排期、不占里程碑;真要提前,只能走"把 clang-p2996 fork 打包成 `llvm-p2996@x` 进生态"的独立打包路径,与本批 issue 无关。 + +--- + +## 9. PR 批次与版本建议 + +``` +0.0.97 里程碑(ffmpeg-m / opencv-m 全源码直编解锁): + PR-1 簇A flag 装配模型(#226 #234) ─┐ 可并行(文件不重叠) + PR-2 簇B 清单语法(#227 #228) ─┘ + PR-3 簇C 构建图身份+depfile(#233 #235) ← 独立,建议先合(#233 阻塞大库直编) + PR-4 簇D 条件源集统一漏斗(#229) ← 依赖清晰,独立 + PR-5 簇E 源发现范围+run缓存(#225) ← 含 #228 glob 入口,若与 PR-2 都改 expand_glob 需约定先后 +并行/独立: + PR-6 簇F 工具供给统一门(#232) + PR-7 簇G workspace 配置与测试基建(#224 + R6 + run成员选择) ← 解锁 index 仓删 smoke shell +复验即关: + #230 windows CI 0.0.96 pin 复验(已由 df985df 修) +不做: + #215 upstream P2996 未落地,无从动手 +``` + +**PR-7 的生态收尾**:合入后到 mcpp-index 侧把 imgui/ffmpeg/opencv 三个模块包各改成 workspace member 测试工程(`default = {path=...}` 一行)、删三份 `smoke_*_module.sh` + 三个专属 CI job,`mcpp test --workspace` 全绿即验收。这是 R6 从"mcpp 补语法"到"index 仓删补丁"的闭环。 + +**合并顺序注意**: +- PR-2(#228 glob 脱糖)与 PR-5(expand_glob 前缀收窄)**都改 `expand_glob` 入口**——建议 **PR-5 先**(把遍历改造做扎实),PR-2 的脱糖叠在其上;或合成一个"scanner glob 引擎"PR。这是唯一的文件级耦合。 +- PR-1 的 `Arg` 规范化落在 scanner 附加处,PR-4 的条件源漏斗落在 prepare per-package 循环,PR-3 落在 plan/ninja_backend——三者**无重叠**,可乱序。 +- **可选精简**:若 reviewer 偏好"按 `[build].flags` 特性端到端",可把 PR-1 + #227 合成一个"[build].flags 管线 PR"(parse 数组表 → normalize include/边界 → escape),把 #228 glob 归 PR-5。本文默认按子系统切,更利于隔离回归。 + +## 10. 待用户 review 的决策点 +1. **版本策略**:A–E 五 PR 是否统一挂 0.0.97 一次发布(vs 分批发)? +2. **PR-1 粒度**:`Arg` 结构化中间表示做到多细(最小两 bit vs 完整类型)?本文建议最小可用、以"边界不丢"为准。 +3. **PR-2/PR-5 合并**:`expand_glob` 改造是拆两 PR(先遍历后脱糖)还是合成一个"scanner glob 引擎"PR? +4. **簇 C depfile 方案**:方案 2(编译边加 `-MD`,根治整类 stale,含额外收益)vs 方案 1(只捞扫描 `.dep`,改动更小但仅覆盖模块 TU)?本文推荐方案 2。 +5. **#227 归属**:独立 PR-2,还是并入"[build].flags 管线"随 PR-1? +6. **R6 默认命名空间重定向拼写**:保留 token `default = { path = ... }`(推荐,可读)、空引号键 `"" = { path = ... }`,还是两者都接受? +7. **PR-7 范围**:#224 + R6 + `run -p member` 三者同 PR(同子系统、无重叠,推荐),还是把 `run -p member` 拆成独立小 PR 先行? diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md new file mode 100644 index 00000000..f5089271 --- /dev/null +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -0,0 +1,453 @@ +# mcpp 0.0.97 架构级修复 —— 实施设计与任务拆分(单 PR / 逐簇 commit) + +> **给实施者**:本计划配套 [2026-07-18-issue-triage-215plus-architectural-remediation.md](2026-07-18-issue-triage-215plus-architectural-remediation.md)(分类与根因)。本文是**可执行计划**:一个大 PR(分支 `feat/v0.0.97-remediation`),**每一类问题一个修复 commit 节点**,末尾一个版本 commit。每个 commit 落地即树保持可编译 + `mcpp test` 绿 + 相关 e2e 绿。步骤用 `- [ ]` 勾选跟踪。 +> +> **目标(Goal)**:以架构级设计(非 workaround)一次性修掉 #224/#225/#226/#227/#228/#229/#232/#233/#234/#235 + R6 + 两条 workspace ergonomics,发布 0.0.97。 +> +> **架构主线(Architecture)**:七簇修复都遵循同一原则——**把结构化意图保留到唯一收敛点,消除"过早压平成串 / 散在多处特判"**:源发现在 glob choke point 收窄并复用缓存;flag 作为 token 列表在规范化+发射两 pass 处理;对象路径镜像源相对路径;条件源集在唯一 per-package 漏斗求值;工具供给复用工具链同款同步门;命名空间路由查表而非硬编码短路。 +> +> **技术栈**:C++23 modules、自研 mcpp(self-host)、ninja backend、自研 TOML parser、gtest 单测(`mcpp test`)、`tests/e2e/*.sh` 端到端。 + +## 全局约束(Global Constraints) + +- **版本**:`MCPP_VERSION` 唯一真源 `src/toolchain/fingerprint.cppm:21`;必须与 `mcpp.toml:3` 同步。本 PR 目标 `0.0.97`(**仅末尾版本 commit 改**,前七个修复 commit 不动版本号)。 +- **基线**:HEAD `42198fb`(0.0.96),分支 `feat/v0.0.97-remediation` 从 main 切出。 +- **每 commit 绿**:任一修复 commit 落地后 `mcpp build`(self-host)+ `mcpp test` 必须通过;涉及行为的簇附带新 e2e(编号从 **114** 起)且并入 `tests/e2e/run_all.sh` 顺序。 +- **单测就近**:新单测加进现有 `tests/unit/test_*.cpp`(gtest,由 `mcpp test` 发现),不新建测试工程。 +- **host-aware**:e2e 断言凡涉及平台差异必须 host-aware(记忆教训:自托管 e2e 会在 win/mac 上跑,勿硬编 Linux 断言)。 +- **closed syntax, open vocabulary**:解析层扩展(#227/#228/R6)只加固定形状机制,不加开放语义键。 +- **无 workaround**:每个修复改的是根因收敛点(见各 commit 的"设计")而非加旁路开关或特判豁免。 + +## Commit 序(DAG)与"每步绿"不变量 + +``` +main(42198fb) + └─ feat/v0.0.97-remediation + C1 [簇E] scanner glob 收窄+边界排除 + run/build 缓存复用 (#225) ← 基础:动 expand_glob + C2 [簇B] TOML 数组表 + glob 花括号交替 (#227 #228) ← 花括号脱糖挂在 C1 后的 expand_glob 入口 + C3 [簇A] flag token 列表:include 族规范化 + 发射端 shell 引号 (#226 #234) + C4 [簇C] 对象路径镜像源相对路径 + 编译边 depfile (#233 #235) ← depfile 依赖对象路径唯一,同 commit 内先做路径 + C5 [簇D] 条件源集统一 per-package 漏斗 (#229) + C6 [簇F] nasm 走工具链同款同步供给门 (#232) + C7 [簇G] workspace 根锚点继承 + 默认命名空间重定向 + run 选成员 (#224 R6 +run-p) + C8 [版本] bump 0.0.97 + CHANGELOG + 发布说明 +``` + +**唯一的 commit 间耦合**:C2 的花括号脱糖复用 C1 重构后的 `expand_glob` 入口 → **C1 必须先于 C2**。其余 C3–C7 相互无文件级冲突(C6/C7 都碰 `prepare.cppm` 但分别在 nasm 段 3019-3032 与命名空间/继承段 577/1134/1297,不重叠),理论可乱序;按上表顺序合入最清晰。#230 已由 df985df 修复,不在本 PR;#215 不做。 + +--- + +## Commit 1 —— 簇 E:源发现收窄 + run/build 缓存复用(#225) + +**Files:** +- Modify: `src/modgraph/scanner.cppm:233-287`(`expand_glob`)、`:289-333`(`expand_dir_glob`) +- Modify: `src/build/execute.cppm:34-41`(`BuildCacheEntry`)、`:43-126`(read/write cache)、`:269-383`(`try_fast_build`)、`:387-431`(`build_run_target`) +- Modify: `src/build/prepare.cppm`(写缓存处 ~`:3100`,补新字段) +- Test: `tests/unit/test_modgraph.cpp`(glob 前缀/边界)、新 `tests/e2e/114_run_scan_scope.sh` + +**Interfaces:** +- Produces:`glob_literal_prefix(std::string_view glob) -> std::filesystem::path`(scanner 内部);`BuildCacheEntry` 新增 `std::vector> runTargets;`(target 名 → 相对 outputDir 的可执行路径)。 +- Consumes:C2 复用 `expand_glob` 的脱糖入口(见 C2)。 + +### 设计(架构,非 workaround) +三处根因(triage §6.2)在**唯一 choke point** `expand_glob`/`expand_dir_glob` 与 run 快路径修: + +1. **前缀收窄遍历**:`expand_glob` 目前恒从 `root` 起 `recursive_directory_iterator`(`scanner.cppm:251-252`),glob 只在遍历后词法过滤。改为先从 glob 提取首个通配符前的字面目录前缀,迭代器从 `root/prefix` 起。 +2. **边界排除**:现仅按名剪 `.mcpp`(`scanner.cppm:261-264`)。扩为一个排除判定:目录名 ∈ {`.mcpp`,`.git`,`target`} 或命中 `/.gitmodules` 登记的子模块路径 → `disable_recursion_pending()`。 +3. **run 复用缓存**:`build_run_target`(`execute.cppm:391`)无条件 `prepare_build`。给它一条与 `cmd_build` 的 `try_fast_build` 平行的快路径:缓存有效(fingerprint + ninja 比源新)则跑 ninja 后从 `runTargets` 定位 exe,跳过 `prepare_build`。为此 `BuildCacheEntry` 持久化 `runTargets`(在 `prepare_build` 成功后连 fingerprint 一起写)。 + +**Before**(`scanner.cppm:251-264` 摘要): +```cpp +fs::recursive_directory_iterator it( + root, fs::directory_options::follow_directory_symlink, ec); +for (...; !ec && it != end; it.increment(ec)) { + auto& e = *it; + if (e.is_directory(eec) && !eec) { + if (e.path().filename() == ".mcpp") { it.disable_recursion_pending(); continue; } + ... +``` +**After**(要点): +```cpp +// 1) 前缀收窄:src/**/*.cppm → 从 root/"src" 起走 +fs::path prefix = glob_literal_prefix(glob); // 首个 * ? { [ 之前的目录段 +fs::path start = prefix.empty() ? root : root / prefix; +if (!fs::exists(start)) return out; // 前缀不存在 = 空集(非全树) +fs::recursive_directory_iterator it( + start, fs::directory_options::follow_directory_symlink, ec); +... + if (e.is_directory(eec) && !eec) { + if (is_excluded_walk_dir(e.path(), root)) { // .mcpp/.git/target/submodule + it.disable_recursion_pending(); continue; + } + ... +``` +> 注:`path_matches_glob` 仍对全 glob 做词法匹配(前缀只收窄**起点**,不改语义);`!` 排除 glob 与 `expand_dir_glob` 同款处理。`glob_literal_prefix` 须对花括号(C2)保守——见 C2 的脱糖先行。 + +### Tasks +- [ ] **Step 1**:`test_modgraph.cpp` 加失败单测 `ExpandGlobStartsAtLiteralPrefix`:构造 `root/{src/a.cppm, junk/big/... }`,断言 `expand_glob(root,"src/**/*.cppm")` 只含 `src/a.cppm` 且**不访问** `junk`(用一个哨兵:在 `junk` 放一个会让旧代码变慢/命中的文件,断言结果集不含它)。再加 `ExcludesGitAndTargetAndMcpp`:`root/{.git/x.cppm, target/y.cppm, .mcpp/z.cppm, src/ok.cppm}` → 只 `src/ok.cppm`。 +- [ ] **Step 2**:`mcpp test`,确认新单测 FAIL(旧代码会把 junk/.git/target 命中或遍历)。 +- [ ] **Step 3**:实现 `glob_literal_prefix` + `is_excluded_walk_dir`(读一次 root `.gitmodules` 缓存子模块路径集),改 `expand_glob` 与 `expand_dir_glob` 两处起点+排除。 +- [ ] **Step 4**:`mcpp test` 绿。 +- [ ] **Step 5**:`BuildCacheEntry` 加 `runTargets`;`write_build_cache`/`read_build_cache` 序列化该字段(向后兼容:旧缓存无此行 → 视为无效,重跑一次);`prepare_build` 成功后填充(从 `ctx.plan.linkUnits` 取每个 bin target 的名与相对 exe 路径)。 +- [ ] **Step 6**:`build_run_target` 加快路径:`read_build_cache` + fingerprint/ninja 新鲜度校验(复用 `try_fast_build` 的校验,并把其硬编码 `src/` 遍历统一到 `expand_glob`)→ 命中则 ninja + 从 `runTargets` 定位 exe;未命中回落 `prepare_build`。 +- [ ] **Step 7**:新增 `tests/e2e/114_run_scan_scope.sh`:建含大无关目录(伪 `compat/blob` 千文件 + `src/main.cpp`)的工程,`mcpp build` 后两次 `mcpp run`,断言第二次 `real` 与缓存 `mcpp build` 同量级(设阈值,如 < 2s;避免绝对时间脆弱,用"run 不重新扫描"标志:`MCPP_VERBOSE` 下断言无 scan 日志行)。并入 `run_all.sh`。 +- [ ] **Step 8**:`mcpp test` + `bash tests/e2e/114_run_scan_scope.sh` 绿。 +- [ ] **Step 9**:commit +```bash +git add src/modgraph/scanner.cppm src/build/execute.cppm src/build/prepare.cppm \ + tests/unit/test_modgraph.cpp tests/e2e/114_run_scan_scope.sh tests/e2e/run_all.sh +git commit -m "perf(scanner): bound glob walk to literal prefix + exclude vcs/build dirs; reuse resolved cache in \`mcpp run\` (#225)" +``` + +--- + +## Commit 2 —— 簇 B:清单语法表达力(#227 数组表 + #228 花括号) + +**Files:** +- Modify: `src/libs/toml.cppm:460-499`(解析主循环,`[` 分支 471-476)——加 `[[table]]` 累积 +- Modify: `src/manifest/toml.cppm:692-730`(`build.flags` 消费,接受数组表形式) +- Modify: `src/modgraph/scanner.cppm`(`expand_glob` 入口加花括号脱糖;`scan_one_into` glob 消费处) +- Test: `tests/unit/test_toml.cpp`(AOT)、`tests/unit/test_modgraph.cpp`(brace)、新 `tests/e2e/115_manifest_syntax_aot_brace.sh` + +**Interfaces:** +- Consumes:C1 的 `expand_glob`(脱糖插在其最前)。 +- Produces:`expand_braces(std::string_view glob) -> std::vector`(scanner 内部,笛卡尔展开 `{a,b}`)。 + +### 设计 +两处纯语法补齐(triage §3.3),零新语义: +1. **#227**:`libs/toml.cppm:474-476` 现见 `[[` 直接报错。改为:识别 `[[path]]` → 在文档里对 `path` 维护一个**表数组**,每遇一次追加一个新子表、后续键值填入该子表。这是给自研 parser 补一个**通用 TOML 能力**。随后 `manifest/toml.cppm:692-730` 让 `build.flags` **同时**接受内联表数组(现状)与数组表(新),声明顺序=应用顺序不变。 +2. **#228**:`expand_braces` 在 `expand_glob` 入口把 `a/{x,y}/**` 展成 `["a/x/**","a/y/**"]`(支持嵌套/多组,笛卡尔积),每分支走既有匹配;`glob_literal_prefix`(C1)对含 `{` 的原始 glob 返回 `{` 前的前缀,脱糖后各分支再各自取更长前缀。 + +### Tasks +- [ ] **Step 1**:`test_toml.cpp` 加 `ParsesArrayOfTables`:解析 +```toml +[[build.flags]] +glob = "a/**" +cxxflags = ["-DX=1"] +[[build.flags]] +glob = "b/**" +``` +断言得到 2 个条目、键值正确、顺序保持。 +- [ ] **Step 2**:`mcpp test` → FAIL(现 `array-of-tables not supported`)。 +- [ ] **Step 3**:实现 libs/toml.cppm 的 AOT 累积 + manifest/toml.cppm 消费两形式。 +- [ ] **Step 4**:`mcpp test` 绿。 +- [ ] **Step 5**:`test_modgraph.cpp` 加 `ExpandBraceAlternation`:`root/{p/aac/x.c, p/bsf/y.c, p/opus/z.c}`,断言 `expand_glob(root,"p/{aac,bsf}/**")` 含 aac/bsf 不含 opus。→ FAIL。 +- [ ] **Step 6**:实现 `expand_braces` 并接到 `expand_glob` 入口 + `scan_one_into` glob 消费点;`glob_literal_prefix` 遇 `{` 保守回退。 +- [ ] **Step 7**:`mcpp test` 绿。 +- [ ] **Step 8**:`tests/e2e/115_manifest_syntax_aot_brace.sh`:一个工程用 `[[build.flags]]` + `{a,b}` glob 编译两目录,断言构建成功、define 生效。并入 `run_all.sh`。 +- [ ] **Step 9**:`mcpp test` + e2e 绿。 +- [ ] **Step 10**:commit +```bash +git commit -m "feat(manifest): support [[build.flags]] array-of-tables + glob brace alternation {a,b} (#227 #228)" +``` + +--- + +## Commit 3 —— 簇 A:flag token 列表规范化 + 发射端引号(#226 #234) + +**Files:** +- Modify: `src/modgraph/scanner.cppm:338-350`(`absolutize_include_flags` → 泛化为 include 族) +- Modify: `src/build/ninja_backend.cppm:99-106`(`join_flags` 加逐 token 引号)、`src/build/flags.cppm:312-331`(全局 blob 装配加同款引号) +- Test: `tests/unit/test_build_flags.cpp`、`tests/unit/test_ninja_backend.cpp`、新 `tests/e2e/116_include_family_and_define_spaces.sh` + +**Interfaces:** +- Produces:`normalize_include_flags(root, std::vector&)`(取代 `absolutize_include_flags`);`shell_quote_arg(std::string_view) -> std::string`(平台感知:POSIX 单引号 / Windows 双引号,叠加 ninja `$` 转义)。 + +### 设计(修正 triage §2.3 的 `Arg` 草案 → 更贴合源码的最小忠实设计) +关键澄清:**flag vector 的每个元素本就 = 一个 argv token**——`apply_glob_flags`(`scanner.cppm:578-580`)push 的 `"-D"+d`(即 `-DT=long long`)是**一个**元素、边界并未丢。#234 的 bug **纯在发射端** `join_flags`(`ninja_backend.cppm:99-106`)裸空格拼接、不引号。故不需要引入新 `Arg` 结构(那是过度设计);正确架构 = **把 vector 当 token 列表对待**,补两个 pass: + +1. **include 族规范化**(#226):`absolutize_include_flags` 只认 `-I`(`scanner.cppm:342` 字面 `starts_with("-I")`)。泛化:一张前缀表 `{-I,-iquote,-isystem,-idirafter,-iprefix,-L}`,每 flag: + - **joined**(`-iquotehdr`):`starts_with(prefix) && len>prefix` → 重写尾部路径; + - **separated**(`-isystem` 后跟独立元素 `hdr`):元素恰等于裸前缀 → 下一元素为待重写路径; + - 重写 = 相对(`!has_root_path()`)则 `root/p`。`-I` 拼写仍走 `Dialect::includePrefix`。把 `flags.cppm:161-165` 那条**另一路** `includeDirs` 绝对化并入同一函数,消除双实现。 +2. **发射端 shell 引号**(#234):`shell_quote_arg` 对含空格/shell 特殊字符的 token 加引号,在 `join_flags`(per-unit)与 `flags.cppm:312-331`(全局 blob)两个 choke point 逐 token 施加,叠加既有 ninja `$`/`:` 转义(`escape_flag_path`)。 + +**Before**(`scanner.cppm:341-348`): +```cpp +for (auto& f : flags) { + if (f.size() > 2 && f.starts_with("-I")) { + std::filesystem::path p(f.substr(2)); + if (!p.has_root_path()) f = "-I" + (root / p).string(); + } +} +``` +**After**(结构): +```cpp +static constexpr std::string_view kIncPrefixes[] = + {"-I","-iquote","-isystem","-idirafter","-iprefix","-L"}; +for (std::size_t i = 0; i < flags.size(); ++i) { + for (auto pre : kIncPrefixes) { + if (flags[i] == pre && i+1 < flags.size()) { // separated + rewrite_rel(flags[i+1], root); ++i; break; + } + if (flags[i].size() > pre.size() && flags[i].starts_with(pre)) { // joined + std::string tail = flags[i].substr(pre.size()); + std::string abs = rewrite_rel_copy(tail, root); + if (abs != tail) flags[i] = std::string(pre) + abs; + break; + } + } +} +``` + +### Tasks +- [ ] **Step 1**:`test_build_flags.cpp` 加 `RewritesFullIncludeFamily`:输入 `["-Ihdr","-iquotehdr","-isystem","hdr","-idirafterhdr"]` + root `/proj`,断言四者路径都变 `/proj/hdr`(joined 拼回前缀、separated 改下一元素)。 +- [ ] **Step 2**:`mcpp test` → FAIL(现只 `-I` 变)。 +- [ ] **Step 3**:实现 `normalize_include_flags`,替换调用点(`scanner.cppm:618-620/633-635`、`plan.cppm:707-708`),并入 `flags.cppm:161-165` includeDirs 绝对化。 +- [ ] **Step 4**:`mcpp test` 绿。 +- [ ] **Step 5**:`test_ninja_backend.cpp` 加 `QuotesFlagValueWithSpace`:token `-DT=long long` 经 `join_flags` 后在命令行是**单参数**(断言输出含 `'-DT=long long'` 或等效引号形式,不含裸 ` long ` 断裂)。 +- [ ] **Step 6**:`mcpp test` → FAIL。 +- [ ] **Step 7**:实现 `shell_quote_arg`,施加于 `join_flags` + `flags.cppm:312-331`。 +- [ ] **Step 8**:`mcpp test` 绿。 +- [ ] **Step 9**:`tests/e2e/116_include_family_and_define_spaces.sh`:`hdr/magic.h` + `-iquotehdr` include 成功;`defines=["T=long long"]` 编译成功且 `typedef T x` 生效。并入 `run_all.sh`。 +- [ ] **Step 10**:`mcpp test` + e2e 绿。 +- [ ] **Step 11**:commit +```bash +git commit -m "fix(flags): rewrite full -I/-iquote/-isystem include family to project root + shell-quote flag values with spaces (#226 #234)" +``` + +--- + +## Commit 4 —— 簇 C:对象路径镜像 + 编译边 depfile(#233 #235) + +**Files:** +- Modify: `src/modgraph/graph.cppm:15-29`(`SourceUnit` 加 `relPath`)、`src/modgraph/scanner.cppm`(scan 时填 `relPath`) +- Modify: `src/build/plan.cppm:426-462`(对象路径方案 + 唯一性断言) +- Modify: `src/build/ninja_backend.cppm:402-435`(cxx_module/cxx_object 加 depfile) +- Test: `tests/unit/test_dyndep.cpp`/`test_ninja_backend.cpp`、新 `tests/e2e/117_object_path_collision.sh`、`118_purview_include_rebuild.sh` + +**Interfaces:** +- Produces:`SourceUnit.relPath`(源相对包根路径,scanner 填);对象路径 `obj///`。 +- 顺序:**先对象路径唯一(#233)后 depfile(#235)**——`$out.d` 依赖 `$out` 唯一,否则 depfile 也撞。 + +### 设计 +1. **#233 对象路径镜像源相对路径**(triage §4.3):`plan.cppm:450-462` 现按 `parent_path().filename()` 折叠必撞。scanner 掌握 root,故在 `SourceUnit` 存 `relPath = fs::relative(path, root)`;plan 镜像目录层级: + **Before**(`plan.cppm:451-462`): + ```cpp + if (basenameCount[fname] > 1) { + auto parentDir = u.path.parent_path().filename().string(); // 只取父目录名 → 撞 + auto prefix = ... sanitize(pkg) + "_" + parentDir; + cu.object = "obj" / prefix / fname; + } else cu.object = "obj" / fname; + ``` + **After**: + ```cpp + // 镜像源相对路径,天然唯一;去掉 basenameCount 启发式 + auto relDir = u.relPath.parent_path(); // a/src, b/src ... + auto prefix = u.packageName.empty() ? fs::path{} : sanitize(u.packageName); + cu.object = fs::path("obj") / prefix / relDir / fname; // obj//a/src/util.o + ``` + 循环后加**唯一性断言**:`std::set` 收集所有 `cu.object`,冲突则返回可诊断错误(列出 colliders),把 ninja "multiple rules" 前移成 mcpp 层错误。 +2. **#235 编译边 depfile**(triage §4.3 方案 2,根治整类 stale):`cxx_module`/`cxx_object` 在非 MSVC 下无任何 depfile(`ninja_backend.cppm:402-435`,`append_deps` 仅 msvc)。仿 nasm 规则(`:463-468`)给二者的 `$cxx` 命令加 `-MMD -MF $out.d`,规则加 `depfile = $out.d` + `deps = gcc`(非 msvc 分支)。这捕获**编译期全部** header/purview/GMF include,不止 #235 的 purview `.inc`,连"改普通头不重编"的潜伏 bug 一并根治。 + **After**(`cxx_object` 非 msvc): + ```cpp + append(std::format( + " command = $cxx $local_includes $cxxflags $unit_cxxflags -MMD -MF $out.d {}\n", + compile_tail)); + append(" description = OBJ $out\n"); + if (!msvcDeps) { append(" deps = gcc\n"); append(" depfile = $out.d\n"); } + else append_deps(); + ``` + `cxx_module` 同理(把 `-MMD -MF $out.d` 插进非 windows 的 BMI-restat `$cxx ...` 调用与 windows 分支命令,规则加 depfile/deps)。 + +### Tasks +- [ ] **Step 1**:`tests/e2e/117_object_path_collision.sh`:`a/src/util.cpp`+`b/src/util.cpp`+`src/main.cpp` 引用两者,`mcpp build` 成功(现 ninja "multiple rules generate")。 +- [ ] **Step 2**:`bash tests/e2e/117...` → FAIL。 +- [ ] **Step 3**:`SourceUnit` 加 `relPath`,scanner 填;plan.cppm 改对象路径为镜像 + 唯一性断言;确认 `.ddi/.dd`(派生自 `cu.object`)自动跟随。 +- [ ] **Step 4**:`bash tests/e2e/117...` 绿 + `mcpp test` 绿(注意 `test_dyndep`/既有 obj 路径断言若硬编旧格式需同步更新)。 +- [ ] **Step 5**:`tests/e2e/118_purview_include_rebuild.sh`:`m.cppm` purview `#include "vals.inc"`;`mcpp run` 得 41 → 改 vals.inc 41→42 → `mcpp run` 得 **42**(现 stale 得 41)。附加一条:改普通 `.h` 也重编。 +- [ ] **Step 6**:`bash tests/e2e/118...` → FAIL(stale)。 +- [ ] **Step 7**:给 cxx_module/cxx_object 加 `-MMD -MF $out.d` + `depfile`/`deps=gcc`(非 msvc)。 +- [ ] **Step 8**:`bash tests/e2e/118...` 绿 + `mcpp test` 绿。两 e2e 并入 `run_all.sh`。 +- [ ] **Step 9**:commit +```bash +git commit -m "fix(build): mirror source relative path in object paths + track compile-edge depfiles for purview/header includes (#233 #235)" +``` + +--- + +## Commit 5 —— 簇 D:条件源集统一 per-package 漏斗(#229) + +**Files:** +- Modify: `src/build/prepare.cppm`:`merge_conditional_build`(`:345-362`)拆分;调用点重排(退役 `:792` 的 sources/flags 半 + `:1714`;新增 per-package 循环 `:2669-2711` 内调用;保留 `:800-802` 条件依赖 root-only) +- Test: 扩 `tests/unit/test_manifest.cpp` 或直接 e2e;新 `tests/e2e/119_dep_cfg_sources.sh` + +**Interfaces:** +- Produces:`merge_conditional_sources_flags(Manifest&, ctx, triple)`(每包调用,只并 cfg 的 sources/flags);条件**依赖**合并留在 root 专属函数。 + +### 设计(triage §5.3) +建立"每包有效源集 = base + cfg + feature、按已解析 target 求值"的**唯一漏斗**,消除 path/git dep 漏评(#229 根因:`merge_conditional_build` 只在 root `:792` 与 version-dep `:1714` 调,path/git 分支 `:2447-2459` 不调)。做法:把 cfg 的 sources/flags 求值折进 feature sources 已有的 per-package 循环(root `:2689` + 每 dep `:2710`),与 `apply` 同点、扫描前;退役两个散点调用;条件依赖(`:800-802`,须在依赖解析前)留 root。 + +### Tasks +- [ ] **Step 1**:`tests/e2e/119_dep_cfg_sources.sh`:`mylib` 带 `[target.'cfg(linux)'.build] sources=["src/impl.cpp"]`,`app` 以 path dep `import mylib;` 调 `impl_value()`;`mcpp build`(现 undefined reference)+ `mcpp test` 双路径。 +- [ ] **Step 2**:`bash tests/e2e/119...` → FAIL(undefined reference)。 +- [ ] **Step 3**:拆 `merge_conditional_build` → `merge_conditional_sources_flags`(per-package)+ 条件依赖 root 函数;per-package 循环内对 `packages[0..n]` 调用;删 `:792` sources/flags 半与 `:1714`;保留 `:800-802`。 +- [ ] **Step 4**:`bash tests/e2e/119...` 绿(build+test 双路径)+ 既有 `108_cfg_conditional_sources.sh`/`85_target_cfg_build_flags.sh`/`100_feature_sources_test_mode.sh` 回归绿(确认无双重合并)。并入 `run_all.sh`。 +- [ ] **Step 5**:commit +```bash +git commit -m "fix(build): evaluate cfg-conditional sources for every package via one per-package funnel (path/git deps + test mode) (#229)" +``` + +--- + +## Commit 6 —— 簇 F:nasm 走工具链同款同步供给门(#232) + +**Files:** +- Modify: `src/build/prepare.cppm:3019-3032`(nasm 消费边:换用 fetcher 门 + 修 guard 吞错) +- Modify: `src/xlings.cppm:1276-1295`(`ensure_nasm` 退役或委托) +- Test: 扩 `tests/unit/test_xlings.cpp`/`test_pm_package_fetcher.cpp`;e2e 复用 `105_asm_sources_nasm.sh`(冷环境变体) + +### 设计(triage §7.3) +nasm 现走 bespoke 弱机制(无索引刷新前置、失败降 warning、`if(cfgNasm)` guard 吞真错)。改为走工具链同款同步门 `Fetcher::resolve_xpkg_path("xim:nasm", autoInstall=true, &progress)`(范式见 `prepare.cppm:851/958-959`,impl `package_fetcher.cppm:644-762`:先刷索引→阻塞 install→校验 payload→XLINGS_HOME 兜底),失败**硬错**;并修 guard:不静默丢 `get_cfg()` 的 bootstrap 错、config 失败时不跳过供给。`xlings::ensure_nasm` 保留为 PATH→sandbox 探测的薄封装或删。 + +**Before**(`prepare.cppm:3019-3032`): +```cpp +auto cfgNasm = get_cfg(); +if (cfgNasm) nasmBin = mcpp::xlings::ensure_nasm(make_xlings_env(**cfgNasm), false, {}); +if (!nasmBin) return std::unexpected("...no usable nasm..."); +``` +**After**(结构): +```cpp +auto cfgNasm = get_cfg(); +if (!cfgNasm) return std::unexpected(cfgNasm.error()); // 不再吞错 +if (auto sys = xlings::find_usable_nasm(**cfgNasm)) nasmBin = *sys; // PATH/sandbox 快路 +else { + Fetcher f(...); // 同工具链构造 + auto p = f.resolve_xpkg_path(nasm_pkg.target(), /*autoInstall=*/true, &progress); + if (!p) return std::unexpected(std::format("nasm provisioning failed: {}", p.error())); + nasmBin = xlings::find_sandbox_nasm(**cfgNasm); +} +if (!nasmBin) return std::unexpected("...no usable nasm..."); +``` + +### Tasks +- [ ] **Step 1**:`test_xlings.cpp`/`test_pm_package_fetcher.cpp` 加断言:nasm 供给路径**调用** `resolve_xpkg_path`(可用一个可注入的 fetcher stub 或验证"索引刷新前置被触发");并断言 `get_cfg()` 错误被**透传**而非吞掉。 +- [ ] **Step 2**:`mcpp test` → FAIL。 +- [ ] **Step 3**:实现 fetcher 门替换 + guard 修复;抽 `find_usable_nasm`(PATH+sandbox)复用。 +- [ ] **Step 4**:`mcpp test` 绿。 +- [ ] **Step 5**:`105_asm_sources_nasm.sh` 加/改冷环境断言:临时空 `MCPP_HOME` + PATH 无 nasm → `mcpp build` 一次成功(供给同步阻塞完成),错误信息若失败须指明真实根因(非"no nasm")。 +- [ ] **Step 6**:`bash tests/e2e/105...` 绿(有 nasm 沙箱的环境)。 +- [ ] **Step 7**:commit +```bash +git commit -m "fix(nasm): provision nasm through the toolchain's synchronous resolve_xpkg_path gate (index refresh + hard error), stop swallowing config bootstrap errors (#232)" +``` + +--- + +## Commit 7 —— 簇 G:workspace 根锚点继承 + 默认命名空间重定向 + run 选成员(#224 R6 + run-p) + +**Files:** +- Modify: `src/project.cppm:52-75`(`merge_workspace_deps` 传播 `path` + 根锚点) +- Modify: `src/build/prepare.cppm:577-579/600-602`(索引继承相对根)、`:1134-1145` + `:1297-1310`(默认命名空间查表) +- Modify: `src/manifest/toml.cppm:924-948`(`default`/`""` 索引键归一化到 `kDefaultNamespace`) +- Modify: `src/cli/cmd_build.cppm:99-107`(`cmd_run` 加 `--package`)、`src/build/execute.cppm:387-431`(`build_run_target` 加 `package_filter`) +- Test: `tests/unit/test_manifest.cpp`;新 `tests/e2e/120_ws_root_indices.sh`、`121_default_ns_redirect.sh`、`122_run_member.sh` + +### 设计(triage §8) +同子系统三洞一并补(可选按 G1/G2/G3 拆三 commit,见文末): +1. **#224 根锚点继承**:`merge_workspace_deps`(`project.cppm:52-75`)现只抄 `version`。改为传播 `path`(记录"相对 workspace 根");`prepare.cppm:577/600` 继承索引时相对**workspace 根**解析 `path`(线程 wsRoot),而非成员目录。 +2. **R6 默认命名空间重定向**:两处硬短路让 `[indices]` 对默认命名空间失效—— + **Before**:`prepare.cppm:1140` `if (ns == kDefaultNamespace) return true;`;`:1300` `if (ns.empty() || ns == kDefaultNamespace) return nullptr;` + **After**:两处在返回 builtin/nullptr **前先查** `m->indices`—— + ```cpp + // usesBuiltinIndex (1140): + if (ns == kDefaultNamespace) { + auto it = m->indices.find(std::string(kDefaultNamespace)); + return it == m->indices.end() || it->second.is_builtin(); + } + // findIndexForNs (1300): + if (ns.empty() || ns == kDefaultNamespace) { + auto it = m->indices.find(std::string(kDefaultNamespace)); + return it == m->indices.end() ? nullptr : &it->second; + } + ``` + `toml.cppm:924-948` 把索引键 `default`(建议规范拼写)或 `""` 归一化为 `kDefaultNamespace` 存入 `m.indices`。 +3. **run 选成员**:`cmd_run`(`cmd_build.cppm:99-107`)接入 `--package` + `resolve_member_dir`(与 build/test 同规则);`build_run_target` 加 `package_filter` 形参,在成员目录 scope 内解析并启动该成员二进制。 + +### Tasks +- [ ] **Step 1**:`test_manifest.cpp` 加 `DefaultNsIndexRedirect`:manifest 带 `[indices] default = {path="x"}` + 默认命名空间 dep,断言 `findIndexForNs("")` 返回该 spec(非 nullptr)。→ FAIL。 +- [ ] **Step 2**:`mcpp test` → FAIL。 +- [ ] **Step 3**:实现 toml 键归一化 + 两处短路查表。 +- [ ] **Step 4**:`mcpp test` 绿。 +- [ ] **Step 5**:`project.cppm` 传播 path + `prepare.cppm` 索引继承相对 wsRoot;`tests/e2e/120_ws_root_indices.sh`:root `[indices] x={path="mcpp"}` 一次声明,成员不重复声明即可解析;path dep 经 `workspace.dependencies` 可用。 +- [ ] **Step 6**:`bash tests/e2e/120...` 绿。 +- [ ] **Step 7**:`tests/e2e/121_default_ns_redirect.sh`:一个"模块包"风格工程(默认命名空间)+ `default={path=本地包}`,`mcpp build && mcpp run` 用本地 checkout 解析(不碰线上 index)。 +- [ ] **Step 8**:`bash tests/e2e/121...` 绿。 +- [ ] **Step 9**:`cmd_run` + `build_run_target` 加成员选择;`tests/e2e/122_run_member.sh`:workspace 两成员各一 bin,`mcpp run -p memberB` 跑 B 的二进制。 +- [ ] **Step 10**:`bash tests/e2e/122...` 绿 + 三 e2e 并入 `run_all.sh` + `90_workspace_test.sh` 回归绿。 +- [ ] **Step 11**:commit +```bash +git commit -m "feat(workspace): root-anchored path/index inheritance + default-namespace index redirect + \`mcpp run -p \` (#224, R6)" +``` + +--- + +## Commit 8 —— 版本 bump 0.0.97 + CHANGELOG + 发布说明 + +**Files:** +- Modify: `src/toolchain/fingerprint.cppm:21`、`mcpp.toml:3`、`CHANGELOG`(若有)、`.agents/docs` 发布说明 + +### Tasks +- [ ] **Step 1**:`MCPP_VERSION` `0.0.96`→`0.0.97`;`mcpp.toml` version 同步。 +- [ ] **Step 2**:`mcpp build`(self-host)成功,`mcpp --version` 打 0.0.97。 +- [ ] **Step 3**:写 CHANGELOG:七簇一句话 + issue 号;注明 #230 已于 0.0.96 修、#215 不做。 +- [ ] **Step 4**:全量 `mcpp test` + `bash tests/e2e/run_all.sh` 绿(host-aware)。 +- [ ] **Step 5**:commit +```bash +git commit -m "chore(release): bump mcpp 0.0.96 -> 0.0.97 (scanner scope, flag model, build-graph identity, cfg funnel, nasm gate, workspace)" +``` +> 发布运维(镜像/索引/pin bump)按 [记忆:release-publish-pipeline] 在 PR 合入后单独走,不在本 PR。**坑复盘**:publish-ecosystem 30min timeout(大件 gtc 补传)、squash `--delete-branch` 会 close 子 PR——本 PR 单分支单 PR 无叠栈,规避该坑。 + +--- + +## PR 描述模板(单 PR) + +``` +title: feat: 0.0.97 架构级修复 —— scanner 范围/flag 模型/构建图身份/条件源漏斗/nasm 供给门/workspace + +本 PR 以架构级设计(非 workaround)一次修掉 #224/#225/#226/#227/#228/#229/#232/#233/#234/#235 + R6, +按"每一类问题一个 commit 修复节点"组织,末尾一个版本 commit。设计见 +.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md。 + +Commits(逐簇): +- 簇E #225 scanner glob 收窄 + 边界排除 + run 复用缓存 +- 簇B #227 #228 TOML 数组表 + glob 花括号 +- 簇A #226 #234 include 族规范化 + 发射端 shell 引号 +- 簇C #233 #235 对象路径镜像 + 编译边 depfile +- 簇D #229 条件源集统一 per-package 漏斗 +- 簇F #232 nasm 走工具链同款同步供给门 +- 簇G #224 R6 workspace 根锚点继承 + 默认命名空间重定向 + run 选成员 +- release bump 0.0.97 + +Closes #224 #225 #226 #227 #228 #229 #232 #233 #234 #235 +Note: #230 已于 0.0.96(df985df)修复,windows CI 0.0.97 pin 复验后一并关闭;#215 不做(挂 upstream P2996)。 +``` + +--- + +## 自审:spec 覆盖对照 + +| triage 簇/issue | 本计划 commit | 覆盖点 | +|---|---|---| +| E #225 | C1 | 前缀收窄 + 边界排除 + run 缓存快路径 ✅ | +| B #227 | C2 | libs/toml AOT + build.flags 消费 ✅ | +| B #228 | C2 | expand_braces 脱糖 ✅ | +| A #226 | C3 | include 族(joined/separated)统一重写 ✅ | +| A #234 | C3 | 发射端逐 token shell 引号 ✅ | +| C #233 | C4 | 对象路径镜像相对路径 + 唯一性断言 ✅ | +| C #235 | C4 | 编译边 -MMD depfile(含普通头额外收益)✅ | +| D #229 | C5 | per-package 条件源漏斗(build+test+path/git dep)✅ | +| F #232 | C6 | fetcher 同步门 + guard 修复 ✅ | +| G #224 | C7 | 根锚点继承 + path 传播 ✅ | +| G R6 | C7 | 默认命名空间查表重定向 ✅ | +| G run-p | C7 | cmd_run 成员选择 ✅ | +| #230 | — | 已 0.0.96 修,仅复验 ✅ | +| #215 | — | 不做 ✅ | + +**类型一致性核对**:`glob_literal_prefix`/`expand_braces`(C1/C2 同在 scanner)、`normalize_include_flags`/`shell_quote_arg`(C3)、`SourceUnit.relPath`(C4 定义、plan 消费)、`merge_conditional_sources_flags`(C5)、`find_usable_nasm`(C6)、`build_run_target(package_filter)`(C7)命名前后一致。 + +## 待实施者/maintainer 决策(继承 triage §10) +1. **簇 G 粒度**:一个 commit(C7,推荐,同子系统)还是拆 G1/G2/G3 三 commit?本计划默认合一。 +2. **R6 索引键拼写**:`default = {path}`(推荐)vs `"" = {path}` vs 两者都接受(归一化点在 `toml.cppm:924-948`)。 +3. **簇 A**:确认采用"token 列表 + 两 pass"的忠实设计(本计划,取代 triage 的 `Arg` 结构草案)。 +4. **执行方式**:subagent-driven(每 commit 一个 fresh subagent + 审查)vs 本会话 inline 批执行。 From 991dc2a74d70efc55536830c9a86a832e1bdf176 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 16:23:11 +0800 Subject: [PATCH 02/21] docs(plan): lock 4 decisions (G=1 commit, R6=default, A=token-list, subagent-driven) + progress tracker --- ...tectural-remediation-implementation-plan.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index f5089271..8bfb4035 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -8,6 +8,24 @@ > > **技术栈**:C++23 modules、自研 mcpp(self-host)、ninja backend、自研 TOML parser、gtest 单测(`mcpp test`)、`tests/e2e/*.sh` 端到端。 +## 决策已定(2026-07-18) +1. 簇 G 粒度 → **一个 commit(C7)**。 2. R6 键拼写 → **`default = {path}`**(归一化点 `toml.cppm:924-948`;`""` 亦归一到 `kDefaultNamespace`,但文档/示例只推 `default`)。 3. 簇 A → **token-list 两 pass**(取代 `Arg` 草案)。 4. 执行 → **subagent-driven**(每 commit 一个 fresh subagent + commit 间审查)。 + +## 实施进度(动态更新) +- 环境:bootstrap `mcpp 0.0.96` @ `/home/speak/.xlings/data/xpkgs/xim-x-mcpp/0.0.96/bin/mcpp`;基线 `mcpp build` 43s、`mcpp test` 35 binaries 全绿。分支 `feat/v0.0.97-remediation`(docs commit 已落)。 + +| commit | 状态 | 备注 | +|---|---|---| +| C1 簇E #225 | ⏳ 进行中 | | +| C2 簇B #227 #228 | ☐ 待办 | | +| C3 簇A #226 #234 | ☐ 待办 | | +| C4 簇C #233 #235 | ☐ 待办 | | +| C5 簇D #229 | ☐ 待办 | | +| C6 簇F #232 | ☐ 待办 | | +| C7 簇G #224 R6 +run-p | ☐ 待办 | | +| C8 版本 0.0.97 | ☐ 待办 | | +| PR / CI / 合入 / release | ☐ 待办 | | + ## 全局约束(Global Constraints) - **版本**:`MCPP_VERSION` 唯一真源 `src/toolchain/fingerprint.cppm:21`;必须与 `mcpp.toml:3` 同步。本 PR 目标 `0.0.97`(**仅末尾版本 commit 改**,前七个修复 commit 不动版本号)。 From 33e6d9a61862bdd38b5f0acf507fa59caa5fd4ed Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 16:52:20 +0800 Subject: [PATCH 03/21] perf(scanner): bound glob walk to literal prefix + exclude vcs/build dirs; reuse resolved cache in `mcpp run` (#225) --- src/build/execute.cppm | 346 +++++++++++++++++++++++++------- src/build/prepare.cppm | 8 + src/modgraph/scanner.cppm | 129 ++++++++++-- tests/e2e/114_run_scan_scope.sh | 97 +++++++++ tests/unit/test_modgraph.cpp | 112 +++++++++++ 5 files changed, 607 insertions(+), 85 deletions(-) create mode 100755 tests/e2e/114_run_scan_scope.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 13fa9d38..0b65c8f4 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -38,6 +38,22 @@ struct BuildCacheEntry { std::string fingerprint; // outputDir basename std::string runtimeEnvKey; // "-" means intentionally empty; "" means old cache std::string runtimeEnvValue; + // mcpp#225 (E2): resolved binary run-targets, cached alongside the + // fingerprint so `mcpp run` can skip prepare_build (toolchain + // resolution + modgraph scan) on a cache hit — see build_run_target's + // fast path. name -> exe path relative to outputDir. Caches written + // before this field existed leave it empty, which the run fast-path + // treats as a miss (falls back to prepare_build once, never crashes). + std::vector> runTargets; + // The process environment needed to exec those targets (e.g. + // LD_LIBRARY_PATH for dep .so's not covered by the exe's own RUNPATH), + // cached the same way as runtimeEnvKey/Value above but for RUNNING the + // binary rather than invoking the toolchain. "" (default-constructed) + // means old cache / not yet resolved — the run fast-path exec's with no + // extra env in that case, matching prepare_build's behavior when + // plan.runtimeLibraryDirs is empty. + std::string runEnvKey; + std::string runEnvValue; }; std::vector read_build_cache(const std::filesystem::path& projectRoot) { @@ -75,11 +91,34 @@ std::vector read_build_cache(const std::filesystem::path& proje if (!std::getline(f, e.ninjaProgram) || e.ninjaProgram.empty()) break; std::getline(f, e.fingerprint); bool haveNextLine = static_cast(std::getline(f, line)); - if (haveNextLine && !line.starts_with("[target=")) { + if (haveNextLine && !line.starts_with("[target=") + && !line.starts_with("runTargets=")) { e.runtimeEnvKey = line; std::getline(f, e.runtimeEnvValue); haveNextLine = static_cast(std::getline(f, line)); } + // mcpp#225 (E2): optional runTargets block. Absent on caches written + // before this field existed (or truncated/corrupt mid-block) — in + // either case e.runTargets stays empty, which the run fast-path + // treats as a miss, never a crash. + if (haveNextLine && line.starts_with("runTargets=")) { + std::size_t n = 0; + try { n = std::stoul(line.substr(11)); } catch (...) { n = 0; } + for (std::size_t i = 0; i < n && std::getline(f, line); ++i) { + auto tab = line.find('\t'); + if (tab == std::string::npos) continue; + e.runTargets.emplace_back(line.substr(0, tab), line.substr(tab + 1)); + } + haveNextLine = static_cast(std::getline(f, line)); + } + // mcpp#225 (E2): optional run-env block (the process env needed to + // exec a cached run-target, e.g. LD_LIBRARY_PATH). Same back-compat + // contract as runTargets above. + if (haveNextLine && line.starts_with("runEnv=")) { + e.runEnvKey = line.substr(7); + std::getline(f, e.runEnvValue); + haveNextLine = static_cast(std::getline(f, line)); + } entries.push_back(std::move(e)); if (!haveNextLine || line.empty()) break; } @@ -92,7 +131,10 @@ void write_build_cache(const std::filesystem::path& projectRoot, const std::string& targetTriple, const std::string& fingerprintHex = "", const std::string& runtimeEnvKey = "-", - const std::string& runtimeEnvValue = "") { + const std::string& runtimeEnvValue = "", + std::vector> runTargets = {}, + const std::string& runEnvKey = "", + const std::string& runEnvValue = "") { auto path = projectRoot / kBuildCacheFile; auto entries = read_build_cache(projectRoot); @@ -103,7 +145,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, // Insert at front (MRU). BuildCacheEntry newEntry{targetTriple, outputDir.string(), ninjaProgram, fingerprintHex, - runtimeEnvKey, runtimeEnvValue}; + runtimeEnvKey, runtimeEnvValue, std::move(runTargets), + runEnvKey, runEnvValue}; entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -122,6 +165,15 @@ void write_build_cache(const std::filesystem::path& projectRoot, f << e.fingerprint << '\n'; f << (e.runtimeEnvKey.empty() ? "-" : e.runtimeEnvKey) << '\n'; f << e.runtimeEnvValue << '\n'; + // mcpp#225 (E2): run-targets + their exec env, always written (even + // when empty) so a reader never has to guess whether a missing + // block means "no targets" vs "cache predates this field" — the + // count-prefixed block is unambiguous either way, and back-compat + // for OLD caches (no such block at all) is handled on the read side. + f << "runTargets=" << e.runTargets.size() << '\n'; + for (auto& [name, exe] : e.runTargets) f << name << '\t' << exe << '\n'; + f << "runEnv=" << e.runEnvKey << '\n'; + f << e.runEnvValue << '\n'; } } @@ -165,6 +217,34 @@ bool is_stale_ninja_failure(std::string_view output) { || output.find("missing and no known rule to make") != std::string_view::npos; } +// mcpp#225 (E2): the (name, exe-path-relative-to-outputDir) pairs for every +// binary link unit in a resolved plan, cached alongside the build +// fingerprint so `mcpp run` can locate an executable without re-running +// prepare_build (see BuildCacheEntry::runTargets / try_fast_run below). +// TestBinary/library link units never run via `mcpp run`, so only Binary +// link units are collected. +std::vector> +compute_run_targets(const mcpp::build::BuildPlan& plan) { + std::vector> out; + for (auto& lu : plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::Binary) continue; + out.emplace_back(lu.targetName, lu.output.generic_string()); + } + return out; +} + +// mcpp#225 (E2): the process env needed to exec a run-target (e.g. +// LD_LIBRARY_PATH for dep .so's not covered by the exe's own RUNPATH). +// Shared between build_run_target's normal (prepare_build) path and its +// cached fast path so both derive the same env from the same source. +std::pair +compute_run_env(const mcpp::build::BuildPlan& plan) { + auto key = mcpp::platform::env::runtime_library_path_key(); + auto value = mcpp::platform::env::prepend_path_list(key, plan.runtimeLibraryDirs); + if (key.empty() || value.empty()) return {"", ""}; + return {key, value}; +} + // Compile a prepared BuildContext. Shared between `mcpp build` and `mcpp run` // so the latter doesn't call prepare_build twice (and re-print the toolchain // resolution banner). @@ -245,10 +325,13 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // P0: save build cache for fast-path on next invocation. if (!no_cache && !r->ninjaProgram.empty()) { auto fpHex = ctx.outputDir.filename().string(); + auto runTargets = compute_run_targets(ctx.plan); + auto [runEnvKey, runEnvValue] = compute_run_env(ctx.plan); write_build_cache(ctx.projectRoot, ctx.outputDir, r->ninjaProgram, std::string(targetOverride), fpHex, r->runtimeEnvKey.empty() ? "-" : r->runtimeEnvKey, - r->runtimeEnvValue); + r->runtimeEnvValue, + std::move(runTargets), runEnvKey, runEnvValue); } mcpp::ui::finished("release", r->elapsed); @@ -264,6 +347,97 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // // This reduces no-change builds from ~10s to <0.5s. +// mcpp#225: is any tracked source file under `projectRoot` newer than +// `ninjaTime`? Shared by try_fast_build's and try_fast_run's freshness +// gates. Uses expand_glob's bounded ("src" prefix) + vcs/build-dir-excluded +// walk instead of a hand-rolled recursive_directory_iterator — the OLD +// staleness check here walked ALL of src/ unfiltered (harmless when src/ is +// the whole tree, but wasteful/wrong the moment a huge unrelated directory +// lives elsewhere under the project root and gets swept in by some other +// caller's broader glob; and it's the same choke-point fix as expand_glob +// itself, see scanner.cppm). +bool sources_newer_than(const std::filesystem::path& projectRoot, + std::filesystem::file_time_type ninjaTime) { + std::error_code ec; + for (auto& f : mcpp::modgraph::expand_glob(projectRoot, "src/**/*")) { + auto ext = f.extension().string(); + if (ext != ".cppm" && ext != ".cpp" && ext != ".cc" && + ext != ".cxx" && ext != ".c" && ext != ".h" && ext != ".hpp") + continue; + auto ft = std::filesystem::last_write_time(f, ec); + if (ec || ft > ninjaTime) return true; + } + return false; +} + +// mcpp#225: run ninja quietly against an already-verified-fresh build.ninja. +// Shared by try_fast_build (which just reports "Finished" on success) and +// try_fast_run (which goes on to locate + exec a binary). Returns nullopt +// when ninja's failure looks like a stale-graph signature — the caller +// should abandon the fast path and fall back to a full prepare_build — or +// an exit code otherwise (0 success; 1 hard failure, diagnostics already +// printed to stderr). +std::optional run_ninja_fast(const std::string& ninjaProgram, + const std::filesystem::path& outputDir, + const std::filesystem::path& ninjaPath, + bool verbose, + const std::string& runtimeEnvKey, + const std::string& runtimeEnvValue, + std::chrono::milliseconds* elapsedOut = nullptr) { + std::vector argv{ninjaProgram}; + if (!verbose) argv.push_back("--quiet"); + argv.push_back("-C"); + argv.push_back(outputDir.string()); + if (verbose) argv.push_back("-v"); + + std::vector> childEnv; + if (runtimeEnvKey == "@env") { + // Multi-var encoding (MSVC INCLUDE/LIB/PATH/VSLANG + optional runtime + // pair): \x1f-separated k=v records in the single value slot. + std::string_view rest = runtimeEnvValue; + while (!rest.empty()) { + auto sep = rest.find('\x1f'); + auto rec = rest.substr(0, sep); + if (auto eq = rec.find('='); eq != std::string_view::npos && eq > 0) + childEnv.emplace_back(std::string(rec.substr(0, eq)), + std::string(rec.substr(eq + 1))); + if (sep == std::string_view::npos) break; + rest.remove_prefix(sep + 1); + } + } else if (runtimeEnvKey != "-" && !runtimeEnvValue.empty()) { + childEnv.emplace_back(runtimeEnvKey, runtimeEnvValue); + } + + auto t0 = std::chrono::steady_clock::now(); + // capture_exec merges stderr into the captured output (replacing `2>&1`), + // so is_stale_ninja_failure / filter_ninja_output still see ninja errors. + auto r = mcpp::platform::process::capture_exec(argv, childEnv); + std::string out = r.output; + int status = r.exit_code; + if (status != 0) { + if (is_stale_ninja_failure(out)) + return std::nullopt; + std::fflush(stdout); + mcpp::ui::error("build failed"); + auto prefixes = read_ninja_command_prefixes(ninjaPath); + auto diagnostics = verbose ? out : mcpp::build::filter_ninja_output(out, prefixes); + if (!diagnostics.empty()) { + std::fputs(diagnostics.c_str(), stderr); + if (diagnostics.back() != '\n') + std::fputc('\n', stderr); + } + return 1; + } + if (verbose && !out.empty()) + std::fputs(out.c_str(), stdout); + + if (elapsedOut) { + *elapsedOut = std::chrono::duration_cast( + std::chrono::steady_clock::now() - t0); + } + return 0; +} + // Try to fast-path: if build.ninja is newer than all inputs, just run ninja. // Returns exit code on fast-path, or nullopt if full rebuild needed. export std::optional try_fast_build(const std::filesystem::path& projectRoot, @@ -313,79 +487,115 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo auto tomlTime = std::filesystem::last_write_time(tomlPath, ec); if (ec || tomlTime > ninjaTime) return std::nullopt; - // Check all source files under src/ - auto srcDir = projectRoot / "src"; - if (std::filesystem::exists(srcDir, ec)) { - for (auto& entry : std::filesystem::recursive_directory_iterator(srcDir, ec)) { - if (!entry.is_regular_file()) continue; - auto ext = entry.path().extension().string(); - if (ext != ".cppm" && ext != ".cpp" && ext != ".cc" && - ext != ".cxx" && ext != ".c" && ext != ".h" && ext != ".hpp") - continue; - auto ft = std::filesystem::last_write_time(entry.path(), ec); - if (ec || ft > ninjaTime) return std::nullopt; - } - } + // mcpp#225: bounded + vcs/build-dir-excluded walk (see sources_newer_than) + // instead of a hand-rolled recursive_directory_iterator over src/. + if (sources_newer_than(projectRoot, ninjaTime)) return std::nullopt; // All inputs are older than build.ninja → fast-path: just run ninja. - std::vector argv{ninjaProgram}; - if (!verbose) argv.push_back("--quiet"); - argv.push_back("-C"); - argv.push_back(outputDir.string()); - if (verbose) argv.push_back("-v"); + std::chrono::milliseconds elapsed{}; + auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, verbose, + runtimeEnvKey, runtimeEnvValue, &elapsed); + if (!rc) return std::nullopt; + if (*rc != 0) return rc; - std::vector> childEnv; - if (runtimeEnvKey == "@env") { - // Multi-var encoding (MSVC INCLUDE/LIB/PATH/VSLANG + optional runtime - // pair): \x1f-separated k=v records in the single value slot. - std::string_view rest = runtimeEnvValue; - while (!rest.empty()) { - auto sep = rest.find('\x1f'); - auto rec = rest.substr(0, sep); - if (auto eq = rec.find('='); eq != std::string_view::npos && eq > 0) - childEnv.emplace_back(std::string(rec.substr(0, eq)), - std::string(rec.substr(eq + 1))); - if (sep == std::string_view::npos) break; - rest.remove_prefix(sep + 1); - } - } else if (runtimeEnvKey != "-" && !runtimeEnvValue.empty()) { - childEnv.emplace_back(runtimeEnvKey, runtimeEnvValue); + mcpp::ui::finished("release", elapsed); + return 0; +} + +// mcpp#225 (E2): `mcpp run`'s fast path. Mirrors try_fast_build's +// fingerprint/freshness gate against the SAME cache entry `mcpp build` +// wrote (targetTriple == "" — `mcpp run` never takes a --target flag), then +// on a hit runs ninja and execs the cached run-target directly — skipping +// prepare_build (toolchain resolution + full modgraph scan) entirely. +// Returns nullopt when there's no usable cache entry (build_run_target +// falls back to the full prepare_build path, which also refreshes the +// cache for next time), an exit code otherwise. +std::optional try_fast_run(const std::filesystem::path& projectRoot, + const std::optional& targetName, + std::span passthrough) { + auto entries = read_build_cache(projectRoot); + const BuildCacheEntry* match = nullptr; + for (auto& e : entries) { + if (e.targetTriple.empty()) { match = &e; break; } } + if (!match || match->runTargets.empty()) return std::nullopt; - auto t0 = std::chrono::steady_clock::now(); - // capture_exec merges stderr into the captured output (replacing `2>&1`), - // so is_stale_ninja_failure / filter_ninja_output still see ninja errors. - auto r = mcpp::platform::process::capture_exec(argv, childEnv); - std::string out = r.output; - int status = r.exit_code; - bool ok = (status == 0); - if (!ok) { - if (is_stale_ninja_failure(out)) - return std::nullopt; - std::fflush(stdout); - mcpp::ui::error("build failed"); - auto prefixes = read_ninja_command_prefixes(ninjaPath); - auto diagnostics = verbose ? out : mcpp::build::filter_ninja_output(out, prefixes); - if (!diagnostics.empty()) { - std::fputs(diagnostics.c_str(), stderr); - if (diagnostics.back() != '\n') - std::fputc('\n', stderr); - } - return 1; + auto outputDirStr = match->outputDir; + auto ninjaProgram = match->ninjaProgram; + // Legacy caches stored a shell-quoted path; execvp needs the raw path. + if (ninjaProgram.size() >= 2 && ninjaProgram.front() == '\'' + && ninjaProgram.back() == '\'') + ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); + if (match->runtimeEnvKey.empty()) + return std::nullopt; // old cache entry; go through prepare_build once + + // P1: verify fingerprint matches the outputDir basename. + if (!match->fingerprint.empty()) { + auto dirBasename = std::filesystem::path(outputDirStr).filename().string(); + if (dirBasename != match->fingerprint) return std::nullopt; } - if (verbose && !out.empty()) - std::fputs(out.c_str(), stdout); - auto elapsed = std::chrono::duration_cast( - std::chrono::steady_clock::now() - t0); - mcpp::ui::finished("release", elapsed); - return 0; + // Locate the requested run-target before doing any filesystem freshness + // work — an unrecognized name falls back to prepare_build, which gives + // a proper "no binary target 'x' found" error instead of a silent miss. + const std::pair* chosen = nullptr; + for (auto& rt : match->runTargets) { + if (targetName && rt.first != *targetName) continue; + chosen = &rt; + if (targetName) break; + } + if (!chosen) return std::nullopt; + + std::error_code ec; + std::filesystem::path outputDir(outputDirStr); + auto ninjaPath = outputDir / "build.ninja"; + if (!std::filesystem::exists(ninjaPath, ec)) return std::nullopt; + auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); + if (ec) return std::nullopt; + + auto tomlPath = projectRoot / "mcpp.toml"; + auto tomlTime = std::filesystem::last_write_time(tomlPath, ec); + if (ec || tomlTime > ninjaTime) return std::nullopt; + + if (sources_newer_than(projectRoot, ninjaTime)) return std::nullopt; + + // Fresh → run ninja (picks up any incremental object/link work) then + // exec the cached exe path directly. + auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, /*verbose=*/false, + match->runtimeEnvKey, match->runtimeEnvValue); + if (!rc) return std::nullopt; + if (*rc != 0) return rc; + + auto exe = outputDir / chosen->second; + auto pathCtx = mcpp::fetcher::make_path_ctx(/*cfg=*/nullptr, projectRoot); + mcpp::ui::status("Running", + std::format("`{}`", mcpp::ui::shorten_path(exe, pathCtx))); + std::println(""); + std::fflush(stdout); + std::vector argv; + argv.push_back(exe.string()); + for (auto& a : passthrough) argv.push_back(a); + + std::vector> childEnv; + if (!match->runEnvKey.empty() && !match->runEnvValue.empty()) + childEnv.emplace_back(match->runEnvKey, match->runEnvValue); + + return mcpp::platform::process::run_exec(argv, childEnv) == 0 ? 0 : 1; } // `mcpp run` driver: build, locate the binary target, exec it with the // resolved runtime environment. export int build_run_target(const std::optional& targetName, std::span passthrough) { + // mcpp#225 (E2): reuse the resolved build cache when it's still fresh, + // skipping prepare_build's toolchain resolution + modgraph scan + // entirely — mirrors cmd_build's try_fast_build fast path. + if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path())) { + if (auto rc = try_fast_run(*root, targetName, passthrough)) { + return *rc; + } + } + // Build first. Single prepare_build → drive build → reuse ctx to locate // the binary, so we don't re-resolve the toolchain or re-scan modgraph. auto ctx = prepare_build(/*print_fp=*/false); @@ -418,11 +628,9 @@ export int build_run_target(const std::optional& targetName, for (auto& a : passthrough) argv.push_back(a); std::vector> childEnv; - auto runtimeEnvKey = mcpp::platform::env::runtime_library_path_key(); - auto runtimeEnvValue = mcpp::platform::env::prepend_path_list( - runtimeEnvKey, ctx->plan.runtimeLibraryDirs); - if (!runtimeEnvKey.empty() && !runtimeEnvValue.empty()) - childEnv.emplace_back(runtimeEnvKey, runtimeEnvValue); + auto [runEnvKey, runEnvValue] = compute_run_env(ctx->plan); + if (!runEnvKey.empty() && !runEnvValue.empty()) + childEnv.emplace_back(runEnvKey, runEnvValue); // Direct exec (no /bin/sh): the loader env reaches ONLY the target child, // never mcpp or a host shell. Fixes the bundled-glibc-vs-host-libtinfo diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 3b08b53a..c338771b 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -11,6 +11,7 @@ export module mcpp.build.prepare; import std; import mcpp.libs.json; +import mcpp.log; import mcpp.manifest; import mcpp.modgraph.graph; import mcpp.modgraph.scanner; @@ -2874,6 +2875,13 @@ prepare_build(bool print_fingerprint, stdFlagAndDialect += f; } + // mcpp#225 (E2): observability marker for the source-discovery phase — + // `mcpp run`'s fast path (build_run_target/try_fast_run in execute.cppm) + // skips prepare_build ENTIRELY on a cache hit, so this line's absence + // under MCPP_VERBOSE=1 on a second `mcpp run` is the "did we re-scan" + // signal the e2e test asserts on (tests/e2e/114_run_scan_scope.sh). + mcpp::log::verbose("scan", "scanning module sources"); + // Modgraph: regex scanner by default; opt-in to compiler-driven P1689 // scanner via env var MCPP_SCANNER=p1689 (see docs/27). auto scan = [&] { diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 04988964..b2c17e83 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -38,6 +38,13 @@ std::vector expand_glob(const std::filesystem::path& root std::vector expand_dir_glob(const std::filesystem::path& root, std::string_view glob); +// mcpp#225 (test-exposed): pure literal (non-wildcard) directory-prefix +// derivation used to bound expand_glob/expand_dir_glob's walk start point. +// Exported (rather than kept file-local) solely so unit tests can assert its +// behavior directly and deterministically — see its definition below for +// the full contract. +std::filesystem::path glob_literal_prefix(std::string_view glob); + // Scan a single source file. std::expected scan_file(const std::filesystem::path& file, const std::string& packageName); @@ -228,14 +235,101 @@ bool is_module_name_char(char c) { return std::isalnum(static_cast(c)) || c == '_' || c == '.' || c == ':'; } +// mcpp#225: submodule paths registered in `/.gitmodules` ("path = ..." +// entries), resolved to canonical absolute paths. is_excluded_walk_dir is +// called once per directory ENTRY seen during a walk, so this is parsed +// once per root and cached for the life of the process rather than +// re-reading .gitmodules on every call (that would defeat the point of +// bounding the walk). +const std::set& +submodule_paths(const std::filesystem::path& root) { + static std::map> cache; + std::error_code kec; + auto key = std::filesystem::canonical(root, kec); + if (kec) key = root; + if (auto it = cache.find(key); it != cache.end()) return it->second; + + std::set paths; + std::ifstream f(root / ".gitmodules"); + std::string line; + while (f && std::getline(f, line)) { + auto eq = line.find('='); + if (eq == std::string::npos) continue; + std::string_view k = trim(std::string_view(line).substr(0, eq)); + if (k != "path") continue; + std::string_view v = trim(std::string_view(line).substr(eq + 1)); + if (v.empty()) continue; + std::error_code pec; + auto abs = std::filesystem::canonical(root / std::filesystem::path(std::string(v)), pec); + paths.insert(pec ? (root / std::filesystem::path(std::string(v))) : abs); + } + return cache.emplace(key, std::move(paths)).first->second; +} + +// mcpp#225: directory names that never hold project sources — VCS metadata, +// mcpp's own build output, and mcpp's own project-metadata dir (mcpp#230: +// `.mcpp`'s xlings data tree holds a symlink back to each path-dep index +// root, so following it walks that entire checkout). A directory whose path +// matches a `.gitmodules`-registered submodule path under `root` is pruned +// too — submodules are foreign trees, often huge, and not part of this +// package's source glob. +bool is_excluded_walk_dir(const std::filesystem::path& dir, + const std::filesystem::path& root) { + auto name = dir.filename().string(); + if (name == ".mcpp" || name == ".git" || name == "target") return true; + auto const& submodules = submodule_paths(root); + if (submodules.empty()) return false; + std::error_code ec; + auto c = std::filesystem::canonical(dir, ec); + return submodules.contains(ec ? dir : c); +} + } // namespace +// mcpp#225: the literal (non-wildcard) directory prefix of a glob, e.g. +// "src/**/*.cppm" -> "src", "tests/**/*.cpp" -> "tests", "*/include" -> "" +// (wildcard already in the first segment). Used to bound the walk's START +// point in expand_glob/expand_dir_glob instead of always walking from root +// and filtering lexically afterward — path_matches_glob still does the full +// lexical match, so this only narrows WHERE the iterator begins, never +// which files can match. Truncates back to the last complete '/' so a +// partial segment (e.g. "src/pre*fix" -> "src/pre") is never mistaken for a +// real directory name. Conservative about '{' (and '?'/'['): brace-expansion +// globs are desugared into multiple plain globs before ever reaching here +// (mcpp#225 cluster E follow-up), so any of those chars seen here is just a +// segment boundary, never something to interpret. Declared in the exported +// namespace above (not kept file-local) purely so unit tests can assert its +// behavior directly, deterministically, and independently of filesystem +// enumeration order — see Scanner.GlobLiteralPrefixDerivation. +std::filesystem::path glob_literal_prefix(std::string_view glob) { + // Every current caller already strips a leading `!` (exclusion globs) + // before calling expand_glob/expand_dir_glob, but strip it here too — + // defense in depth, and a literal '!' is never a real path component. + if (!glob.empty() && glob.front() == '!') glob.remove_prefix(1); + auto wildcard = glob.find_first_of("*?{["); + std::string_view literal = wildcard == std::string_view::npos + ? glob : glob.substr(0, wildcard); + auto slash = literal.find_last_of('/'); + if (slash == std::string_view::npos) return {}; + return std::filesystem::path(literal.substr(0, slash)); +} + std::vector expand_glob(const std::filesystem::path& root, std::string_view glob) { namespace fs = std::filesystem; std::vector out; if (!fs::exists(root)) return out; + + // mcpp#225: bound the walk's start point to the glob's literal + // directory prefix instead of always walking the whole root. A prefix + // that doesn't exist means the glob can never match anything — return + // empty WITHOUT walking (not a full-tree fallback). + fs::path prefix = glob_literal_prefix(glob); + fs::path start = prefix.empty() ? root : root / prefix; + std::error_code startEc; + if (!fs::exists(start, startEc)) return out; + // Follow directory symlinks (vendored trees are often symlink farms). // Cycle guard: a directory whose canonical path is already on the // CURRENT recursion chain is a link loop — only that is pruned; the same @@ -245,20 +339,15 @@ std::vector expand_glob(const std::filesystem::path& root std::vector chain; // canonical dirs of the recursion stack std::error_code ec, eec; // ec: iteration; eec: per-entry probes { - auto c = fs::canonical(root, eec); - chain.push_back(eec ? root : c); + auto c = fs::canonical(start, eec); + chain.push_back(eec ? start : c); } fs::recursive_directory_iterator it( - root, fs::directory_options::follow_directory_symlink, ec); + start, fs::directory_options::follow_directory_symlink, ec); for (fs::recursive_directory_iterator end; !ec && it != end; it.increment(ec)) { auto& e = *it; if (e.is_directory(eec) && !eec) { - // `.mcpp` is mcpp's own project metadata dir. Its xlings data - // tree holds a SYMLINK back to each path-dep index root, so - // following it walks that entire checkout (mcpp#230: the CI - // workspace walked into a vendored xim-pkgindex and died on a - // CJK filename). Never a source dir — prune by name. - if (e.path().filename() == ".mcpp") { + if (is_excluded_walk_dir(e.path(), root)) { it.disable_recursion_pending(); continue; } @@ -299,23 +388,31 @@ std::vector expand_dir_glob(const std::filesystem::path& if (std::filesystem::is_directory(p, ec)) out.push_back(p); return out; } - // Walk all directories under root, match each against the glob. Same + + // mcpp#225: bound the walk's start point the same way expand_glob does + // (see the comment there) — a prefix that doesn't exist means the glob + // can never match, so return empty without walking. + std::filesystem::path prefix = glob_literal_prefix(glob); + std::filesystem::path start = prefix.empty() ? root : root / prefix; + std::error_code startEc; + if (!std::filesystem::exists(start, startEc)) return out; + + // Walk all directories under start, match each against the glob. Same // follow-symlinks + recursion-chain cycle guard as expand_glob above. - out.push_back(root); // root itself eligible if glob is "" (rare) + out.push_back(root); // sentinel, always dropped below regardless of value std::vector chain; std::error_code eec; // per-entry probes; ec drives iteration { - auto c = std::filesystem::canonical(root, eec); - chain.push_back(eec ? root : c); + auto c = std::filesystem::canonical(start, eec); + chain.push_back(eec ? start : c); } std::filesystem::recursive_directory_iterator it( - root, std::filesystem::directory_options::follow_directory_symlink, ec); + start, std::filesystem::directory_options::follow_directory_symlink, ec); for (std::filesystem::recursive_directory_iterator end; !ec && it != end; it.increment(ec)) { auto& e = *it; if (!e.is_directory(eec) || eec) continue; - // Same `.mcpp` prune as expand_glob (see comment there / mcpp#230). - if (e.path().filename() == ".mcpp") { + if (is_excluded_walk_dir(e.path(), root)) { it.disable_recursion_pending(); continue; } diff --git a/tests/e2e/114_run_scan_scope.sh b/tests/e2e/114_run_scan_scope.sh new file mode 100755 index 00000000..0e337cc2 --- /dev/null +++ b/tests/e2e/114_run_scan_scope.sh @@ -0,0 +1,97 @@ +#!/usr/bin/env bash +# requires: gcc +# mcpp#225 (cluster E — E1 bounded glob walk + E2 run-cache reuse): +# E1: expand_glob must bound its walk to the glob's literal directory +# prefix ("src" for "src/**/*.{...}") instead of always walking the +# whole project root. A large, unrelated top-level sibling directory +# (`blob/`, ~1500 files) exercises exactly the tree an unbounded walk +# would otherwise have to stat/canonicalize on every scan. +# E2: `mcpp run` must reuse the build cache `mcpp build` already resolved +# (fingerprint + ninja freshness, same gate as `mcpp build`'s own +# fast path) instead of unconditionally calling prepare_build (full +# toolchain resolution + modgraph scan) on every invocation. +# +# Not asserted via wall-clock (brittle across CI hosts) — asserted via the +# absence of the scan phase's observability marker: prepare_build logs +# `mcpp::log::verbose("scan", "scanning module sources")` right before it +# invokes the scanner (see prepare.cppm). Under MCPP_VERBOSE=1 that line is +# grep-able on stderr. `mcpp run` only reaches prepare_build when its own +# fast path (try_fast_run in execute.cppm) misses — so if the line is +# absent, the run never re-scanned. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +mkdir -p proj/src proj/blob + +cat > proj/mcpp.toml <<'EOF' +[package] +name = "runscope" +version = "0.1.0" +EOF + +cat > proj/src/main.cpp <<'EOF' +#include +int main() { std::puts("hello from runscope"); return 0; } +EOF + +# Large unrelated sibling tree, NOT under src/ and not matched by the +# default "src/**/*.{...}" source glob — present purely to make an +# unbounded (root-wide) walk expensive/observable, and to prove the bounded +# walk (E1) never has a reason to touch it. +for i in $(seq 1 1500); do + : > "proj/blob/file_$i.txt" +done + +cd proj +"$MCPP" build + +# `mcpp build` already resolves + writes a complete cache (incl. +# BuildCacheEntry::runTargets) via run_build_plan, so even the FIRST +# `mcpp run` after a build should hit the fast path — there is no separate +# "warm-up" run required. +MCPP_VERBOSE=1 "$MCPP" run > "$TMP/run1.out" 2>&1 +if grep -q 'scan: scanning module sources' "$TMP/run1.out"; then + echo "FAIL: first mcpp run (post-build) re-scanned module sources instead of reusing the build cache" + cat "$TMP/run1.out" + exit 1 +fi +grep -q "hello from runscope" "$TMP/run1.out" || { + echo "FAIL: mcpp run did not produce expected program output" + cat "$TMP/run1.out" + exit 1 +} + +# Second run: same assertion, now also covering that the fast path is +# stable across repeated invocations (not a one-shot fluke). +MCPP_VERBOSE=1 "$MCPP" run > "$TMP/run2.out" 2>&1 +if grep -q 'scan: scanning module sources' "$TMP/run2.out"; then + echo "FAIL: second mcpp run re-scanned module sources instead of reusing the build cache" + cat "$TMP/run2.out" + exit 1 +fi +grep -q "hello from runscope" "$TMP/run2.out" || { + echo "FAIL: mcpp run did not produce expected program output" + cat "$TMP/run2.out" + exit 1 +} + +# Sanity: touching a source file must still force a real rebuild (the fast +# path's freshness gate, not a permanent bypass). +sleep 1.1 +touch src/main.cpp +MCPP_VERBOSE=1 "$MCPP" run > "$TMP/run3.out" 2>&1 +grep -q 'scan: scanning module sources' "$TMP/run3.out" || { + echo "FAIL: mcpp run did not re-scan after a source change (fast path should have missed)" + cat "$TMP/run3.out" + exit 1 +} +grep -q "hello from runscope" "$TMP/run3.out" || { + echo "FAIL: mcpp run did not produce expected program output after rebuild" + cat "$TMP/run3.out" + exit 1 +} + +echo "PASS: mcpp run reuses the resolved build cache (no re-scan) with a large unrelated sibling tree present, and still rebuilds on source changes" diff --git a/tests/unit/test_modgraph.cpp b/tests/unit/test_modgraph.cpp index 6fbbb23e..cfdcdabf 100644 --- a/tests/unit/test_modgraph.cpp +++ b/tests/unit/test_modgraph.cpp @@ -140,6 +140,118 @@ TEST(Scanner, ExpandGlobFollowsDirectorySymlinks) { std::filesystem::remove_all(dir); } +// mcpp#225 review finding: glob_literal_prefix's contract (the piece that +// actually decides WHERE expand_glob/expand_dir_glob start walking) is the +// thing that must be locked down deterministically. Calling the pure helper +// directly — rather than inferring its behavior from a filesystem walk whose +// entry order is unspecified by the standard — makes this test's pass/fail +// independent of directory-enumeration order on any filesystem. +TEST(Scanner, GlobLiteralPrefixDerivation) { + EXPECT_EQ(glob_literal_prefix("src/**/*.cppm"), "src"); + // Wildcard already in the first segment: no literal directory to bound to. + EXPECT_EQ(glob_literal_prefix("**/*.c"), ""); + // No wildcard at all: the full parent directory path is the prefix. + EXPECT_EQ(glob_literal_prefix("a/b/c.cpp"), "a/b"); + // Truncate back to the last COMPLETE '/' before the first wildcard char — + // "x*.cpp" is a partial segment, not a real directory named "x". + EXPECT_EQ(glob_literal_prefix("src/x*.cpp"), "src"); + // '{' is treated as a segment-boundary wildcard char (brace-expansion + // globs are desugared before reaching this helper — see its comment in + // scanner.cppm), so the prefix truncates at the last '/' before it. + EXPECT_EQ(glob_literal_prefix("a/{x,y}/z"), "a"); +} + +// mcpp#225: expand_glob must bound its walk to the glob's literal directory +// prefix ("src" for "src/**/*.cppm") instead of always walking the whole +// root and lexically filtering afterward. This is the FUNCTIONAL half of the +// regression guard: given a normal (non-adversarial) tree with files outside +// "src", a bounded walk returns exactly the "src" matches. The core "walk +// starts AT the literal prefix, not root" behavior is now locked down +// deterministically by Scanner.GlobLiteralPrefixDerivation above, which +// tests glob_literal_prefix directly and isn't subject to filesystem +// enumeration order. +TEST(Scanner, ExpandGlobStartsAtLiteralPrefix) { + auto dir = make_tempdir("mcpp-scanner-prefix"); + write(dir / "other" / "b.cppm", "export module b;\n"); + write(dir / "src" / "a.cppm", "export module a;\n"); + + auto files = expand_glob(dir, "src/**/*.cppm"); + + ASSERT_EQ(files.size(), 1u); + EXPECT_EQ(files[0], dir / "src" / "a.cppm"); + + std::filesystem::remove_all(dir); +} + +// Supplementary (not the primary regression guard — see +// Scanner.GlobLiteralPrefixDerivation and Scanner.ExpandGlobStartsAtLiteralPrefix +// above): on filesystems where directory enumeration happens to visit "junk" +// before "src", this additionally proves the walk never even touches an +// unreadable sibling tree. The landmine's fire/no-fire outcome depends on +// unspecified recursive_directory_iterator enumeration order, so the guard +// condition is made explicit below rather than silently no-op'ing — if the +// landmine didn't fire (order-dependent), the test still asserts the +// positive result and reports (via trace) that the landmine was inert this +// run, instead of pretending it verified the old-code-fails claim. +TEST(Scanner, ExpandGlobStartsAtLiteralPrefixLandmine) { + auto dir = make_tempdir("mcpp-scanner-prefix-landmine"); + auto blocked = dir / "junk" / "blocked"; + std::filesystem::create_directories(blocked); + write(blocked / "sentinel.cppm", "export module sentinel;\n"); + std::error_code permEc; + std::filesystem::permissions(blocked, std::filesystem::perms::none, permEc); + + write(dir / "src" / "a.cppm", "export module a;\n"); + + // If permissions can't be locked down here (e.g. running as root), the + // landmine can't fire — skip rather than risk a false pass/fail. + std::error_code probeEc; + std::filesystem::directory_iterator(blocked, probeEc); + if (!probeEc) { + std::filesystem::permissions(blocked, std::filesystem::perms::all, permEc); + std::filesystem::remove_all(dir); + GTEST_SKIP() << "cannot restrict directory permissions in this environment"; + } + + auto files = expand_glob(dir, "src/**/*.cppm"); + + // Restore permissions before cleanup (remove_all needs to read `blocked`). + std::filesystem::permissions(blocked, std::filesystem::perms::all, permEc); + + // This is the guard being made explicit: whether the landmine actually + // fired for the OLD (unbounded) code depends on unspecified enumeration + // order, so it cannot be asserted here either way. What IS asserted, + // unconditionally, is the functional outcome — the bounded walk must + // return exactly the "src" match regardless of "junk"'s enumeration + // position. That functional assertion is real signal on every run; it + // just isn't, by itself, an order-independent proof that old code would + // have failed here (that proof now lives in + // Scanner.GlobLiteralPrefixDerivation instead). + ASSERT_EQ(files.size(), 1u); + EXPECT_EQ(files[0], dir / "src" / "a.cppm"); + + std::filesystem::remove_all(dir); +} + +// mcpp#225: the walk must prune VCS metadata (.git) and mcpp's own build +// output (target) the same way it already prunes .mcpp (mcpp#230). Glob has +// no literal prefix ("**/*.cppm") so the walk starts at root and would +// otherwise reach all three excluded trees. +TEST(Scanner, ExcludesGitAndTargetAndMcpp) { + auto dir = make_tempdir("mcpp-scanner-exclude"); + write(dir / ".git" / "x.cppm", "export module x;\n"); + write(dir / "target" / "y.cppm", "export module y;\n"); + write(dir / ".mcpp" / "z.cppm", "export module z;\n"); + write(dir / "src" / "ok.cppm", "export module ok;\n"); + + auto files = expand_glob(dir, "**/*.cppm"); + + ASSERT_EQ(files.size(), 1u); + EXPECT_EQ(files[0], dir / "src" / "ok.cppm"); + + std::filesystem::remove_all(dir); +} + TEST(Scanner, RecordsPackageLocalIncludeDirs) { auto dir = make_tempdir("mcpp-scanner-includes"); write(dir / "src" / "foo.cpp", From 10edb40e9c343d7d0047c41b91f0673fdfe7c1da Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 17:09:34 +0800 Subject: [PATCH 04/21] =?UTF-8?q?docs(plan):=20C1=20(=E7=B0=87E=20#225)=20?= =?UTF-8?q?done=20=E2=80=94=20scanner=20scope=20+=20run=20cache?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...8-v0.0.97-architectural-remediation-implementation-plan.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index 8bfb4035..e06df604 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -16,8 +16,8 @@ | commit | 状态 | 备注 | |---|---|---| -| C1 簇E #225 | ⏳ 进行中 | | -| C2 簇B #227 #228 | ☐ 待办 | | +| C1 簇E #225 | ✅ 完成 `33e6d9a` | scanner 前缀收窄+边界排除;run 复用缓存(try_fast_run);+2 单测(含确定性 glob_literal_prefix)+e2e 114;review 通过 | +| C2 簇B #227 #228 | ⏳ 进行中 | | | C3 簇A #226 #234 | ☐ 待办 | | | C4 簇C #233 #235 | ☐ 待办 | | | C5 簇D #229 | ☐ 待办 | | From 223087215a49fd7f2667add0433d90282e141496 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 17:22:41 +0800 Subject: [PATCH 05/21] feat(manifest): support [[build.flags]] array-of-tables + glob brace alternation {a,b} (#227 #228) --- src/libs/toml.cppm | 57 ++++++++- src/manifest/toml.cppm | 60 +++++++++- src/modgraph/scanner.cppm | 132 ++++++++++++++++++++- tests/e2e/115_manifest_syntax_aot_brace.sh | 78 ++++++++++++ tests/unit/test_manifest.cpp | 65 ++++++++++ tests/unit/test_modgraph.cpp | 80 +++++++++++++ tests/unit/test_toml.cpp | 50 ++++++++ 7 files changed, 514 insertions(+), 8 deletions(-) create mode 100755 tests/e2e/115_manifest_syntax_aot_brace.sh diff --git a/src/libs/toml.cppm b/src/libs/toml.cppm index 24855d3c..0a1a80d6 100644 --- a/src/libs/toml.cppm +++ b/src/libs/toml.cppm @@ -399,6 +399,43 @@ inline std::string join_path(const std::vector& path) { }); } +// `[[dotted.path]]` — array-of-tables. Navigate to the parent of the last +// path segment (creating intermediate tables as needed, same as a regular +// `[table]` header), then get-or-create an Array at the last segment and +// append a fresh Table to it. Returns a pointer to the newly-appended table +// so the caller can point current_table at it; every subsequent `key = +// value` line lands in THAT table until the next `[...]`/`[[...]]` header. +// Note: nested AOT (e.g. `[[a.b]]` followed by `[[a.b.c]]`, appending inside +// the *last* `a.b` entry) is NOT supported — every `[[dotted.path]]` dives +// from the document root, not from the currently-open AOT entry. Acceptable +// today since mcpp only uses one flat array-of-tables, `[[build.flags]]`. +inline std::expected open_array_of_tables( + Table& root, const std::vector& path, Position pos) +{ + std::vector parents(path.begin(), path.end() - 1); + Table* parent = dive(root, parents); + if (!parent) { + return std::unexpected(ParseError{ + std::format("table path '{}' conflicts with non-table value", + join_path(path)), + pos}); + } + const std::string& last = path.back(); + auto it = parent->find(last); + if (it == parent->end()) { + (*parent)[last] = Value{Array{}}; + it = parent->find(last); + } else if (!it->second.is_array()) { + return std::unexpected(ParseError{ + std::format("table path '{}' conflicts with non-array value", + join_path(path)), + pos}); + } + Array& arr = it->second.as_array(); + arr.push_back(Value{Table{}}); + return &arr.back().as_table(); +} + } // namespace detail const Value* Document::get(std::string_view dotted_path) const { @@ -470,9 +507,25 @@ std::expected parse(std::string_view src) { if (L.peek() == '[') { L.advance(); - // Note: we don't currently distinguish [[array of tables]]; mcpp doesn't use them. if (L.peek() == '[') { - return std::unexpected(ParseError{"array-of-tables not supported", L.position()}); + // [[dotted.path]] — array-of-tables (#227). Every occurrence + // appends a fresh table to the array at `path`; current_table + // then points at that new table for subsequent key = value + // lines. + L.advance(); + auto path = read_dotted_key(L); + if (!path) return std::unexpected(path.error()); + L.skip_inline_whitespace(); + if (L.peek() != ']' || L.peek(1) != ']') { + return std::unexpected(ParseError{"expected ']]'", L.position()}); + } + L.advance(); + L.advance(); + auto t = open_array_of_tables(root, *path, L.position()); + if (!t) return std::unexpected(t.error()); + explicitTables.insert(join_path(*path)); + current_table = *t; + continue; } auto path = read_dotted_key(L); if (!path) return std::unexpected(path.error()); diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 0d776c4b..0414dd27 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -34,6 +34,42 @@ ManifestError error(const std::filesystem::path& origin, return ManifestError{msg, origin, pos.line, pos.column}; } +// #227 follow-up: libs/toml.cppm's `[[dotted.path]]` support is intentionally +// schema-agnostic — it accepts an array-of-tables at ANY dotted path, so a +// doubled-bracket typo like `[[dependencies]]` (single brackets meant) or +// `[[toolchain]]` parses cleanly as an Array Value there. Every mcpp consumer +// reads such sections via get_table(), which returns nullptr for a non-table +// Value, so the section silently reads as ABSENT (e.g. all dependencies +// silently dropped) with no parse error and no warning. Close the grammar +// here, at the manifest layer, instead of hardcoding mcpp section names into +// the generic TOML layer: the only legitimate array-of-tables in mcpp.toml +// today is `[[build.flags]]`. +bool is_array_of_tables(const t::Value& v) { + if (!v.is_array()) return false; + auto& arr = v.as_array(); + if (arr.empty()) return false; + for (auto& e : arr) if (!e.is_table()) return false; + return true; +} + +std::optional find_disallowed_array_of_tables( + const t::Table& tbl, const std::string& prefix, + std::span allowlist) +{ + for (auto& [k, v] : tbl) { + std::string path = prefix.empty() ? k : std::format("{}.{}", prefix, k); + if (is_array_of_tables(v)) { + bool allowed = false; + for (auto a : allowlist) if (a == path) { allowed = true; break; } + if (!allowed) return path; + } else if (v.is_table()) { + if (auto found = find_disallowed_array_of_tables(v.as_table(), path, allowlist)) + return found; + } + } + return std::nullopt; +} + } // namespace std::expected parse_string(std::string_view content, @@ -43,6 +79,17 @@ std::expected parse_string(std::string_view content, return std::unexpected(error(origin, doc.error().message, doc.error().where)); } + // Closed-grammar guard: reject any array-of-tables whose dotted path + // isn't explicitly allowlisted, BEFORE any section is read. See + // find_disallowed_array_of_tables above. + static constexpr std::string_view kAllowedArraysOfTables[] = { "build.flags" }; + if (auto badPath = find_disallowed_array_of_tables(doc->root(), "", kAllowedArraysOfTables)) { + return std::unexpected(error(origin, std::format( + "[[{}]] (array-of-tables) is not allowed for section '{}'; " + "array-of-tables syntax is only supported for [[build.flags]]", + *badPath, *badPath))); + } + Manifest m; m.sourcePath = origin; @@ -689,11 +736,20 @@ std::expected parse_string(std::string_view content, // are sorted maps, so only the array form can carry declaration order, // and order is the override semantics (later entries win via "last flag // wins"). Unknown keys are errors: closed grammar. + // + // #227: `[[build.flags]]` array-of-tables is accepted here too, with no + // extra branching needed — libs/toml.cppm's AOT support + // (open_array_of_tables) builds the exact same shape at this dotted path + // (an Array of Table Values) that the inline form `flags = [{...}, ...]` + // does, so `doc->get("build.flags")` and the loop below see one + // representation regardless of which spelling was used in the source. + // Declaration order is preserved by both (Array is a vector). if (auto* fv = doc->get("build.flags")) { if (!fv->is_array()) { return std::unexpected(error(origin, - "[build].flags must be an array of inline tables: " - "flags = [{ glob = \"...\", cxxflags = [...] }, ...]")); + "[build].flags must be an array of inline tables " + "(flags = [{ glob = \"...\", cxxflags = [...] }, ...]) " + "or an array of tables ([[build.flags]] glob = \"...\")")); } for (auto& ev : fv->as_array()) { if (!ev.is_table()) { diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index b2c17e83..578de337 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -45,6 +45,16 @@ std::vector expand_dir_glob(const std::filesystem::path& // the full contract. std::filesystem::path glob_literal_prefix(std::string_view glob); +// mcpp#228: desugar brace alternation `{a,b}` into a cartesian product of +// plain globs, e.g. "a/{x,y}/**" -> ["a/x/**", "a/y/**"]. A glob with no `{` +// returns itself unchanged (the common case). Multiple and nested groups are +// supported ("a/{x,y}/{1,2}" -> 4 branches; "{a,{b,c}}" resolves the inner +// group too). Exported so unit tests can assert its behavior directly; also +// used internally at the entry of expand_glob and at the [build].flags +// per-glob-flag match point in scan_one_into, so every glob consumer sees +// alternation transparently. +std::vector expand_braces(std::string_view glob); + // Scan a single source file. std::expected scan_file(const std::filesystem::path& file, const std::string& packageName); @@ -314,8 +324,81 @@ std::filesystem::path glob_literal_prefix(std::string_view glob) { return std::filesystem::path(literal.substr(0, slash)); } -std::vector expand_glob(const std::filesystem::path& root, - std::string_view glob) +// mcpp#228: `{a,b}` alternation, recursively. Finds the first top-level `{`, +// its MATCHING `}` (brace-depth tracked, so a nested group's inner braces +// don't prematurely close the outer one), splits the interior on top-level +// commas (again depth-tracked, so a nested group's own commas don't split +// the outer one), then cartesian-products: each alternative is itself +// re-expanded (handles nesting, e.g. "{a,{b,c}}"), and everything AFTER the +// closing `}` is independently re-expanded and combined with every +// alternative (handles multiple groups, e.g. "a/{x,y}/{1,2}" -> 4 +// branches). An unbalanced `{` (no matching `}`) is passed through as a +// literal — never a reason to fail the whole glob. +std::vector expand_braces(std::string_view glob, int depthGuard); + +std::vector expand_braces(std::string_view glob) { + return expand_braces(glob, 0); +} + +std::vector expand_braces(std::string_view glob, int depthGuard) { + auto open = glob.find('{'); + if (open == std::string_view::npos) return { std::string(glob) }; + + // Bound brace-nesting recursion depth: a pathological manifest with + // deeply nested `{` should not stack-overflow. Beyond the cap, treat the + // remainder as a literal passthrough rather than throwing — an + // unreasonably-nested glob simply won't desugar further, it still + // parses. + constexpr int kMaxBraceDepth = 32; + if (depthGuard >= kMaxBraceDepth) return { std::string(glob) }; + + int depth = 0; + std::size_t close = std::string_view::npos; + for (std::size_t i = open; i < glob.size(); ++i) { + if (glob[i] == '{') ++depth; + else if (glob[i] == '}') { + --depth; + if (depth == 0) { close = i; break; } + } + } + if (close == std::string_view::npos) return { std::string(glob) }; + + std::string_view prefix = glob.substr(0, open); + std::string_view inner = glob.substr(open + 1, close - open - 1); + std::string_view suffix = glob.substr(close + 1); + + std::vector alts; + int d = 0; + std::size_t start = 0; + for (std::size_t i = 0; i < inner.size(); ++i) { + if (inner[i] == '{') ++d; + else if (inner[i] == '}') --d; + else if (inner[i] == ',' && d == 0) { + alts.push_back(inner.substr(start, i - start)); + start = i + 1; + } + } + alts.push_back(inner.substr(start)); + + auto suffixBranches = expand_braces(suffix, depthGuard + 1); + std::vector out; + for (auto alt : alts) { + for (auto& altBranch : expand_braces(alt, depthGuard + 1)) { + for (auto& sufBranch : suffixBranches) { + out.push_back(std::string(prefix) + altBranch + sufBranch); + } + } + } + return out; +} + +namespace { + +// mcpp#225: the actual bounded recursive-directory walk for a SINGLE plain +// glob (no `{` — expand_glob below desugars brace alternation via +// expand_braces and calls this once per branch, unioning the results). +std::vector expand_glob_one(const std::filesystem::path& root, + std::string_view glob) { namespace fs = std::filesystem; std::vector out; @@ -375,6 +458,31 @@ std::vector expand_glob(const std::filesystem::path& root return out; } +} // namespace + +std::vector expand_glob(const std::filesystem::path& root, + std::string_view glob) +{ + // mcpp#228: desugar `{a,b}` alternation FIRST, then run the existing + // bounded walk per branch and union+dedup — each post-desugar branch has + // no `{`, so glob_literal_prefix's prefix narrowing still applies (and + // gets a LONGER, more specific prefix per branch than the pre-desugar + // glob would have yielded). + auto branches = expand_braces(glob); + if (branches.size() == 1) return expand_glob_one(root, branches.front()); + + namespace fs = std::filesystem; + std::set seen; + std::vector out; + for (auto const& b : branches) { + for (auto& p : expand_glob_one(root, b)) { + if (seen.insert(p).second) out.push_back(std::move(p)); + } + } + std::sort(out.begin(), out.end()); + return out; +} + std::vector expand_dir_glob(const std::filesystem::path& root, std::string_view glob) { @@ -382,7 +490,10 @@ std::vector expand_dir_glob(const std::filesystem::path& std::error_code ec; if (!std::filesystem::exists(root, ec)) return out; - // Fast path: glob with no wildcards → literal path under root. + // Fast path: glob with no wildcards → literal path under root. Brace + // alternation `{a,b}` is intentionally NOT desugared here (unlike + // expand_glob) — include_dirs entries are meant to name one literal + // directory each; a caller wanting alternatives lists multiple entries. if (glob.find('*') == std::string_view::npos) { auto p = root / std::filesystem::path(glob); if (std::filesystem::is_directory(p, ec)) out.push_back(p); @@ -665,10 +776,23 @@ void scan_one_into(ScanResult& result, // a cfg-gated source set can legitimately leave a glob empty on some // targets. std::vector globFlagHits(manifest.buildConfig.globFlags.size(), 0); + // mcpp#228: pre-desugar each flag-entry's glob once (not per-file) — + // apply_glob_flags below matches via path_matches_glob directly (it + // isn't a filesystem walk like expand_glob, so there's no walk to bound + // a start point for), so a brace glob here needs its OR-of-branches + // matched explicitly rather than picking up desugaring for free. + std::vector> globFlagBranches; + globFlagBranches.reserve(manifest.buildConfig.globFlags.size()); + for (auto const& gf : manifest.buildConfig.globFlags) + globFlagBranches.push_back(expand_braces(gf.glob)); auto apply_glob_flags = [&](SourceUnit& u) { for (std::size_t i = 0; i < manifest.buildConfig.globFlags.size(); ++i) { auto const& gf = manifest.buildConfig.globFlags[i]; - if (!path_matches_glob(u.path, root, gf.glob)) continue; + bool matched = false; + for (auto const& branch : globFlagBranches[i]) { + if (path_matches_glob(u.path, root, branch)) { matched = true; break; } + } + if (!matched) continue; ++globFlagHits[i]; // defines reach asm units too — via the -D subset the backend // filters out of packageCflags (no third copy needed here). diff --git a/tests/e2e/115_manifest_syntax_aot_brace.sh b/tests/e2e/115_manifest_syntax_aot_brace.sh new file mode 100755 index 00000000..580af7d3 --- /dev/null +++ b/tests/e2e/115_manifest_syntax_aot_brace.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# requires: gcc +# #227 + #228: [[build.flags]] array-of-tables and glob brace alternation +# {a,b} are both pure-syntax additions to the manifest grammar — this test +# exercises them TOGETHER, the way a real "codec backends" manifest would: +# - two source directories selected by ONE glob with brace alternation +# (p/{aac,bsf}/**), a third sibling directory (p/opus) excluded from it +# - per-glob flags declared via the NEW [[build.flags]] array-of-tables +# syntax (rather than the existing flags = [{...}] inline-table array), +# with a brace glob on the flags entry itself +# Assert: build succeeds, and the per-glob define only reaches units under +# the brace-matched directories (not the sibling excluded from both the +# sources glob and the flags glob). +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +"$MCPP" new aotbrace > /dev/null +cd aotbrace + +mkdir -p src/p/aac src/p/bsf src/p/opus + +cat > src/p/aac/aac.cpp <<'EOF' +#ifndef CODEC +#error "per-glob define (via [[build.flags]] AOT) did not reach aac.cpp" +#endif +extern "C" int aac_id() { return 1; } +EOF + +cat > src/p/bsf/bsf.cpp <<'EOF' +#ifndef CODEC +#error "per-glob define (via [[build.flags]] AOT) did not reach bsf.cpp" +#endif +extern "C" int bsf_id() { return 2; } +EOF + +cat > src/p/opus/opus.cpp <<'EOF' +#ifdef CODEC +#error "brace glob over-matched: opus.cpp is outside {aac,bsf}" +#endif +extern "C" int opus_id() { return 3; } +EOF + +cat > src/main.cpp <<'EOF' +import std; +extern "C" int aac_id(); +extern "C" int bsf_id(); +extern "C" int opus_id(); +int main() { + std::println("ids = {}", aac_id() + bsf_id() + opus_id()); + return 0; +} +EOF + +# sources: brace alternation selects p/aac and p/bsf, plus opus explicitly +# (so it's compiled but must NOT receive the CODEC define) and main.cpp. +# [build].flags: NEW `[[build.flags]]` array-of-tables syntax (issue #227), +# with a brace glob (issue #228) on the entry itself. +cat > mcpp.toml <<'EOF' +[package] +name = "aotbrace" +version = "0.1.0" + +[build] +sources = ["src/p/{aac,bsf}/**/*.cpp", "src/p/opus/**/*.cpp", "src/main.cpp"] + +[[build.flags]] +glob = "src/p/{aac,bsf}/**" +defines = ["CODEC"] +EOF + +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "build failed"; exit 1; } +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "ids = 6" ]] || { echo "unexpected output: $out (want ids = 6)"; exit 1; } + +echo "OK" diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 83b653e9..270e1194 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -1731,3 +1731,68 @@ mcpp = { EXPECT_EQ(mx->buildConfig.globFlags[1].defines, (std::vector{"HAVE_AVX2"})); } + +TEST(Manifest, AllowsBuildFlagsArrayOfTablesSyntax) { + // `[[build.flags]]` header form must still parse at the manifest layer + // (not just the lower libs/toml layer) — the closed-grammar guard must + // allowlist exactly this path. + constexpr auto src = R"( +[package] +name = "globf" +version = "1.0.0" + +[[build.flags]] +glob = "third_party/**" +cxxflags = ["-w"] + +[[build.flags]] +glob = "src/x86/**/*.asm" +asmflags = ["-DPIC"] +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + auto& gfs = m->buildConfig.globFlags; + ASSERT_EQ(gfs.size(), 2u); + EXPECT_EQ(gfs[0].glob, "third_party/**"); + EXPECT_EQ(gfs[0].cxxflags, (std::vector{"-w"})); + EXPECT_EQ(gfs[1].glob, "src/x86/**/*.asm"); + EXPECT_EQ(gfs[1].asmflags, (std::vector{"-DPIC"})); +} + +// Closed-grammar guard (review fix on #227/#228): `[[dependencies]]` is a +// doubled-bracket typo for `[dependencies]`. Before this guard it parsed +// silently as an array-of-tables at that path; every consumer reads +// [dependencies] via get_table(), which returns nullptr for a non-table +// Value, so the whole section (all dependencies) silently vanished with no +// error and no warning. It must now be a loud parse error. +TEST(Manifest, RejectsArrayOfTablesTypoOnDependencies) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "x" +version = "0.1.0" + +[[dependencies]] +gtest = "1.15.2" +)"); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().message.find("array-of-tables"), std::string::npos) + << m.error().message; + EXPECT_NE(m.error().message.find("dependencies"), std::string::npos) + << m.error().message; +} + +// Same guard, a different non-allowlisted path — confirms the check isn't +// hardcoded to just "dependencies". +TEST(Manifest, RejectsArrayOfTablesTypoOnToolchain) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "x" +version = "0.1.0" + +[[toolchain]] +linux = "gcc@15.1.0" +)"); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().message.find("array-of-tables"), std::string::npos) + << m.error().message; +} diff --git a/tests/unit/test_modgraph.cpp b/tests/unit/test_modgraph.cpp index cfdcdabf..5b104dc0 100644 --- a/tests/unit/test_modgraph.cpp +++ b/tests/unit/test_modgraph.cpp @@ -608,6 +608,86 @@ TEST(Scanner, PerGlobFlagsAttachToMatchedUnits) { std::filesystem::remove_all(dir); } +// #228: expand_braces desugars `{a,b}` into a cartesian product of plain +// globs. No braces -> passthrough; multiple/nested groups combine. +TEST(Scanner, ExpandBracesDesugarsCartesianProduct) { + EXPECT_EQ(expand_braces("a/**"), (std::vector{"a/**"})); + EXPECT_EQ(expand_braces("a/{x,y}/**"), + (std::vector{"a/x/**", "a/y/**"})); + EXPECT_EQ(expand_braces("a/{x,y}/{1,2}"), + (std::vector{"a/x/1", "a/x/2", "a/y/1", "a/y/2"})); + // Nested group. + EXPECT_EQ(expand_braces("a/{x,{y,z}}/**"), + (std::vector{"a/x/**", "a/y/**", "a/z/**"})); +} + +// Review fix on #228: brace-nesting recursion depth is bounded (cap: 32 +// levels), so a pathological manifest with deeply nested `{` cannot +// stack-overflow. Beyond the cap the remainder is passed through as a +// literal instead of throwing — this must simply return, not crash. +TEST(Scanner, ExpandBracesBoundsDeeplyNestedRecursion) { + std::string glob = "a/"; + constexpr int kDepth = 500; // far beyond the 32-level cap + for (int i = 0; i < kDepth; ++i) glob += "{"; + glob += "x"; + for (int i = 0; i < kDepth; ++i) glob += "}"; + + std::vector out; + ASSERT_NO_THROW(out = expand_braces(glob)); + EXPECT_FALSE(out.empty()); +} + +// #228: expand_glob applies brace desugaring at its entry, so a glob like +// "p/{aac,bsf}/**" matches files under EITHER alternative directory and +// nothing else (a sibling directory outside the brace group, "opus", must +// not appear). +TEST(Scanner, ExpandGlobBraceAlternation) { + auto dir = make_tempdir("mcpp-scanner-brace"); + write(dir / "p" / "aac" / "x.c", "int x;\n"); + write(dir / "p" / "bsf" / "y.c", "int y;\n"); + write(dir / "p" / "opus" / "z.c", "int z;\n"); + + auto files = expand_glob(dir, "p/{aac,bsf}/**"); + + ASSERT_EQ(files.size(), 2u); + std::set names; + for (auto& f : files) names.insert(f.filename().string()); + EXPECT_TRUE(names.contains("x.c")); + EXPECT_TRUE(names.contains("y.c")); + EXPECT_FALSE(names.contains("z.c")); + + std::filesystem::remove_all(dir); +} + +// #228: brace alternation in a [build].flags glob must also match — the +// per-glob-flags match point (scan_one_into's apply_glob_flags) uses +// path_matches_glob directly rather than expand_glob's walk, so it needs its +// own desugaring wire-up. +TEST(Scanner, PerGlobFlagsMatchBraceAlternation) { + auto dir = make_tempdir("mcpp-scanner-globflags-brace"); + write(dir / "p" / "aac" / "x.cpp", "int x() { return 1; }\n"); + write(dir / "p" / "bsf" / "y.cpp", "int y() { return 2; }\n"); + write(dir / "p" / "opus" / "z.cpp", "int z() { return 3; }\n"); + + mcpp::manifest::Manifest m; + m.package.name = "bracepkg"; + m.modules.sources = { "p/**/*.cpp" }; + m.buildConfig.globFlags.push_back( + { .glob = "p/{aac,bsf}/**", .defines = {"CODEC"} }); + + auto res = scan_package(dir, m); + ASSERT_TRUE(res.errors.empty()); + ASSERT_EQ(res.graph.units.size(), 3u); + for (auto& u : res.graph.units) { + bool wantsDefine = u.path.filename() != "z.cpp"; + bool hasDefine = std::find(u.packageCxxflags.begin(), u.packageCxxflags.end(), + "-DCODEC") != u.packageCxxflags.end(); + EXPECT_EQ(hasDefine, wantsDefine) << u.path.string(); + } + + std::filesystem::remove_all(dir); +} + // G8b: relative -I flags are root-relative in the manifest but ninja runs // with cwd = output dir — the scanner absolutizes them on every unit. TEST(Scanner, RelativeIncludeFlagsAbsolutized) { diff --git a/tests/unit/test_toml.cpp b/tests/unit/test_toml.cpp index a18c6713..86b35702 100644 --- a/tests/unit/test_toml.cpp +++ b/tests/unit/test_toml.cpp @@ -113,3 +113,53 @@ TEST(Toml, EmptyStringStillParses) { EXPECT_EQ(d->get_string("e").value_or("?"), ""); EXPECT_EQ(d->get_int("x").value_or(-1), 2); } + +// #227: `[[dotted.path]]` array-of-tables — each occurrence appends a fresh +// table to an array at that dotted path; keys after it fill THAT table until +// the next `[...]`/`[[...]]` header. Declaration order must be preserved +// (it's the "later entries win" override order for [build].flags). +TEST(Toml, ParsesArrayOfTables) { + auto d = t::parse(R"( +[[build.flags]] +glob = "a/**" +cxxflags = ["-DX=1"] +[[build.flags]] +glob = "b/**" +)"); + ASSERT_TRUE(d.has_value()) << d.error().message; + auto* v = d->get("build.flags"); + ASSERT_NE(v, nullptr); + ASSERT_TRUE(v->is_array()); + auto& arr = v->as_array(); + ASSERT_EQ(arr.size(), 2u); + ASSERT_TRUE(arr[0].is_table()); + EXPECT_EQ(arr[0].as_table().at("glob").as_string(), "a/**"); + auto cx = arr[0].as_table().at("cxxflags"); + ASSERT_TRUE(cx.is_array()); + ASSERT_EQ(cx.as_array().size(), 1u); + EXPECT_EQ(cx.as_array()[0].as_string(), "-DX=1"); + ASSERT_TRUE(arr[1].is_table()); + EXPECT_EQ(arr[1].as_table().at("glob").as_string(), "b/**"); + EXPECT_EQ(arr[1].as_table().count("cxxflags"), 0u); +} + +// Regular `[table]` and inline-array-of-tables parsing must be unaffected by +// AOT support — array-of-tables is an ADDITIONAL grammar form, not a +// replacement. +TEST(Toml, RegularTablesStillWorkAlongsideArrayOfTables) { + auto d = t::parse(R"( +[package] +name = "x" +[[build.flags]] +glob = "a/**" +[dependencies] +foo = "1.0" +)"); + ASSERT_TRUE(d.has_value()) << d.error().message; + EXPECT_EQ(d->get_string("package.name").value_or(""), "x"); + EXPECT_EQ(d->get_string("dependencies.foo").value_or(""), "1.0"); + auto* v = d->get("build.flags"); + ASSERT_NE(v, nullptr); + ASSERT_TRUE(v->is_array()); + EXPECT_EQ(v->as_array().size(), 1u); +} From 69b8b33809144bdf0602fc813e6252a3ef545631 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 17:43:00 +0800 Subject: [PATCH 06/21] =?UTF-8?q?docs(plan):=20C2=20(=E7=B0=87B=20#227=20#?= =?UTF-8?q?228)=20done=20=E2=80=94=20manifest=20syntax=20+=20AOT=20guard?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...8-v0.0.97-architectural-remediation-implementation-plan.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index e06df604..0b731587 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -17,8 +17,8 @@ | commit | 状态 | 备注 | |---|---|---| | C1 簇E #225 | ✅ 完成 `33e6d9a` | scanner 前缀收窄+边界排除;run 复用缓存(try_fast_run);+2 单测(含确定性 glob_literal_prefix)+e2e 114;review 通过 | -| C2 簇B #227 #228 | ⏳ 进行中 | | -| C3 簇A #226 #234 | ☐ 待办 | | +| C2 簇B #227 #228 | ✅ 完成 `2230872` | libs/toml AOT + expand_braces;consumer 零改(同 Value 形);+manifest 层 closed-grammar 守卫(拒非白名单 `[[x]]` typo,防静默丢数据)+brace 深度上限;+7 单测+e2e 115;review 通过 | +| C3 簇A #226 #234 | ⏳ 进行中 | | | C4 簇C #233 #235 | ☐ 待办 | | | C5 簇D #229 | ☐ 待办 | | | C6 簇F #232 | ☐ 待办 | | From b71d93ced31b21a8e8bb1ad3f4e91b338e80ba41 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 18:02:14 +0800 Subject: [PATCH 07/21] fix(flags): rewrite full -I/-iquote/-isystem include family to project root + shell-quote flag values with spaces (#226 #234) --- src/build/flags.cppm | 74 +++++++++++++++++- src/build/ninja_backend.cppm | 9 ++- src/build/plan.cppm | 4 +- src/modgraph/scanner.cppm | 78 +++++++++++++------ .../116_include_family_and_define_spaces.sh | 73 +++++++++++++++++ tests/unit/test_build_flags.cpp | 38 +++++++++ tests/unit/test_ninja_backend.cpp | 70 +++++++++++++++++ 7 files changed, 317 insertions(+), 29 deletions(-) create mode 100755 tests/e2e/116_include_family_and_define_spaces.sh diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 78ac9731..594b2704 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -13,6 +13,7 @@ export module mcpp.build.flags; import std; import mcpp.build.plan; +import mcpp.modgraph.scanner; import mcpp.platform; import mcpp.toolchain.clang; import mcpp.toolchain.detect; @@ -66,6 +67,21 @@ CompileFlags compute_flags(const BuildPlan& plan); std::string atomic_link_flag(const std::vector& linkDirs, bool staticLink); +// mcpp#234: quote a single flag-vector token for safe embedding in a shell +// command line. Every element of a flags `vector` is already one +// argv token (e.g. `apply_glob_flags` pushes `"-D" + d`, so a define like +// `T=long long` arrives as the single element `-DT=long long`) — but the +// emission choke points (`join_flags` in ninja_backend.cppm, and the global +// blob assembly below) historically joined tokens with a bare space and no +// quoting, so a token containing a space silently split into two shell +// words once ninja handed the resolved command line to the shell. Only +// tokens that actually contain whitespace or a shell-significant character +// are quoted — plain framework flags (`-std=c++23`, `-O2`, `-I/abs/path`) +// come back unchanged, byte-for-byte. POSIX: wrap in single quotes (embedded +// `'` escaped as `'\''`). Windows: wrap in double quotes (embedded `"` +// escaped as `\"`) — cmd.exe/CreateProcess argv convention. +std::string shell_quote_arg(std::string_view arg); + } // namespace mcpp::build namespace mcpp::build { @@ -123,6 +139,39 @@ std::string atomic_link_flag(const std::vector& linkDirs, return {}; } +std::string shell_quote_arg(std::string_view arg) { + // Characters that split/alter a word when unquoted in POSIX sh or + // cmd.exe: whitespace plus the common shell metacharacters. Anything + // NOT in this set (e.g. `-std=c++23`, `-O2`, `-I/abs/path`, `-DFOO=1`) + // returns untouched — no quoting where none is needed. + constexpr std::string_view kNeedsQuote = " \t\n\"'\\$`;&|<>()*?[]#~!{}"; + if (arg.find_first_of(kNeedsQuote) == std::string_view::npos) + return std::string(arg); + + if constexpr (mcpp::platform::is_windows) { + // cmd.exe / CreateProcess argv convention: wrap in double quotes, + // escape embedded `"` as `\"`. + std::string out = "\""; + for (char c : arg) { + if (c == '"') out += "\\\""; + else out.push_back(c); + } + out += "\""; + return out; + } else { + // POSIX sh: wrap in single quotes (nothing is special inside single + // quotes except `'` itself), escaping an embedded `'` as `'\''` + // (close quote, literal quote, reopen quote). + std::string out = "'"; + for (char c : arg) { + if (c == '\'') out += "'\\''"; + else out.push_back(c); + } + out += "'"; + return out; + } +} + CompileFlags compute_flags(const BuildPlan& plan) { CompileFlags f; @@ -157,11 +206,28 @@ CompileFlags compute_flags(const BuildPlan& plan) { } std::string pic_flag = (need_pic && !isMsvcDialect) ? " -fPIC" : ""; - // Include dirs - std::string include_flags; + // Include dirs — this is the TYPED PATH channel (bare paths from the + // manifest; the dialect prefix is applied here at emission), not the + // FLAG-STRING channel that `normalize_include_flags` serves (cflags/ + // cxxflags, where the -I/-iquote/... prefix is already embedded in the + // string by the scanner). `normalize_include_flags`'s prefix table only + // knows GNU spellings, so routing dialect-prefixed tokens through it + // silently no-ops under MSVC (`/Iinclude` matches nothing and is never + // rewritten against plan.projectRoot — but ninja runs with cwd = output + // dir, so a relative include dir stops resolving). Absolutize the path + // directly instead (dialect-agnostic), then prepend the prefix, then + // ninja-$-escape and shell-quote per token (#234) so an include dir + // whose name contains a space can't silently split into two shell words + // once ninja hands the resolved command line to the shell. + std::vector includeTokens; for (auto& inc : plan.manifest.buildConfig.includeDirs) { - auto abs = inc.is_absolute() ? inc : (plan.projectRoot / inc); - include_flags += std::format(" {}{}", d.includePrefix, escape_path(abs)); + std::filesystem::path p = inc.has_root_path() ? inc : (plan.projectRoot / inc); + includeTokens.push_back(std::string(d.includePrefix) + p.string()); + } + std::string include_flags; + for (auto& t : includeTokens) { + include_flags += ' '; + include_flags += shell_quote_arg(escape_path(std::filesystem::path(t))); } // Sysroot / payload paths — resolved ONCE by the toolchain link model diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 36573ed5..f08211ef 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -97,10 +97,17 @@ std::string local_include_flags(const CompileUnit& cu) { } std::string join_flags(const std::vector& flags) { + // mcpp#234: each vector element is already one argv token (e.g. a + // manifest define `T=long long` arrives here as the single element + // `-DT=long long`, pushed whole by apply_glob_flags) — but joining with + // a bare space and no quoting let the embedded space split it into two + // words once ninja handed the resolved command line to the shell. + // shell_quote_arg is a no-op for tokens with nothing shell-significant + // (`-std=c++23`, `-O2`, ...), so plain flags are untouched. std::string out; for (auto const& flag : flags) { out += ' '; - out += flag; + out += shell_quote_arg(flag); } return out; } diff --git a/src/build/plan.cppm b/src/build/plan.cppm index cef187cc..c148c864 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -704,8 +704,8 @@ BuildPlan make_plan(const mcpp::manifest::Manifest& manifest, } // Root-relative -I flags → absolute (G8b), mirroring the scanner's // treatment of every scanned unit. - mcpp::modgraph::absolutize_include_flags(projectRoot, main_cu.packageCflags); - mcpp::modgraph::absolutize_include_flags(projectRoot, main_cu.packageCxxflags); + mcpp::modgraph::normalize_include_flags(projectRoot, main_cu.packageCflags); + mcpp::modgraph::normalize_include_flags(projectRoot, main_cu.packageCxxflags); // We didn't scan main.cpp earlier (it's not in scanner output unless globbed in). // Best-effort: scan its imports here. diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 578de337..8703d39d 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -68,13 +68,18 @@ struct ScanResult { ScanResult scan_package(const std::filesystem::path& root, const mcpp::manifest::Manifest& manifest); -// Absolutize relative `-I` compile flags against the package root -// (G8b). A manifest's relative -I means root-relative, but ninja runs -// commands with cwd = the output dir, so a verbatim relative flag resolves -// against the wrong base. Called at every point where per-unit flag vectors -// are attached (the scanner here; plan.cppm for a target's entry unit). -void absolutize_include_flags(const std::filesystem::path& root, - std::vector& flags); +// Absolutize relative include/lib-search-path flags against the package root +// (G8b, generalized by #226). A manifest's relative include flag means +// root-relative, but ninja runs commands with cwd = the output dir, so a +// verbatim relative flag resolves against the wrong base. Recognizes the +// whole include-family prefix set — -I, -iquote, -isystem, -idirafter, +// -iprefix, -L — in BOTH the joined spelling (`-iquotehdr`) and the +// separated spelling (`-isystem` followed by a standalone `hdr` element). +// Called at every point where per-unit flag vectors are attached (the +// scanner here; plan.cppm for a target's entry unit; flags.cppm for the +// manifest-global [build] include_dirs). +void normalize_include_flags(const std::filesystem::path& root, + std::vector& flags); enum class DependencyVisibility { Private, @@ -543,16 +548,45 @@ std::vector expand_dir_glob(const std::filesystem::path& return out; } -void absolutize_include_flags(const std::filesystem::path& root, - std::vector& flags) +namespace { + +// has_root_path: leave absolute AND root-relative ("/x" on Windows) +// spellings alone — only genuinely root-less paths are project-relative. +std::string rewrite_rel_copy(const std::string& p, const std::filesystem::path& root) { + std::filesystem::path fp(p); + if (fp.has_root_path()) return p; + return (root / fp).string(); +} + +void rewrite_rel(std::string& p, const std::filesystem::path& root) { + p = rewrite_rel_copy(p, root); +} + +} // namespace + +void normalize_include_flags(const std::filesystem::path& root, + std::vector& flags) { - for (auto& f : flags) { - if (f.size() > 2 && f.starts_with("-I")) { - std::filesystem::path p(f.substr(2)); - // has_root_path: leave absolute AND root-relative ("/x" on - // Windows) spellings alone — only genuinely root-less paths are - // project-relative. - if (!p.has_root_path()) f = "-I" + (root / p).string(); + // #226: the whole include/lib-search-path family, not just -I. Each + // prefix is checked in both spellings: + // joined: "-iquotehdr" (element starts_with prefix, has a tail) + // separated: "-isystem", "hdr" (element == bare prefix, rewrite next) + static constexpr std::string_view kIncPrefixes[] = + {"-I", "-iquote", "-isystem", "-idirafter", "-iprefix", "-L"}; + + for (std::size_t i = 0; i < flags.size(); ++i) { + for (auto pre : kIncPrefixes) { + if (flags[i] == pre && i + 1 < flags.size()) { // separated + rewrite_rel(flags[i + 1], root); + ++i; + break; + } + if (flags[i].size() > pre.size() && flags[i].starts_with(pre)) { // joined + std::string tail = flags[i].substr(pre.size()); + std::string abs = rewrite_rel_copy(tail, root); + if (abs != tail) flags[i] = std::string(pre) + abs; + break; + } } } } @@ -836,9 +870,9 @@ void scan_one_into(ScanResult& result, u.packageCflags = packageCflags; u.packageCxxflags = packageCxxflags; apply_glob_flags(u); - absolutize_include_flags(root, u.packageCflags); - absolutize_include_flags(root, u.packageCxxflags); - absolutize_include_flags(root, u.packageAsmflags); + normalize_include_flags(root, u.packageCflags); + normalize_include_flags(root, u.packageCxxflags); + normalize_include_flags(root, u.packageAsmflags); result.graph.units.push_back(std::move(u)); continue; } @@ -851,9 +885,9 @@ void scan_one_into(ScanResult& result, r->packageCflags = packageCflags; r->packageCxxflags = packageCxxflags; apply_glob_flags(*r); - absolutize_include_flags(root, r->packageCflags); - absolutize_include_flags(root, r->packageCxxflags); - absolutize_include_flags(root, r->packageAsmflags); + normalize_include_flags(root, r->packageCflags); + normalize_include_flags(root, r->packageCxxflags); + normalize_include_flags(root, r->packageAsmflags); result.graph.units.push_back(std::move(*r)); } diff --git a/tests/e2e/116_include_family_and_define_spaces.sh b/tests/e2e/116_include_family_and_define_spaces.sh new file mode 100755 index 00000000..edc24da9 --- /dev/null +++ b/tests/e2e/116_include_family_and_define_spaces.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# requires: gcc +# mcpp#226 + mcpp#234: the full include-family flag spelling (-I/-iquote/ +# -isystem/-idirafter/-iprefix/-L) must root-relativize, not just -I; and any +# flag-vector token that contains a space (e.g. `-DT=long long`, one manifest +# `defines` entry) must survive as ONE shell argument through emission, not +# silently split into two. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +"$MCPP" new incfam > /dev/null +cd incfam + +# --- Part 1: #226 — -iquote (joined spelling) resolves root-relative ------- +# hdr/magic.h is NOT next to the including .c file, so plain quote-include +# lookup (current-dir-of-includer, then quote search path) only finds it if +# -iquotehdr got rewritten to an absolute, root-relative "-iquote/hdr". +mkdir -p hdr +cat > hdr/magic.h <<'EOF' +#define MAGIC 42 +EOF + +cat > src/magic_user.c <<'EOF' +#include "magic.h" +int get_magic(void) { return MAGIC; } +EOF + +cat > src/main.cpp <<'EOF' +import std; +extern "C" int get_magic(); +extern long long get_big(); + +int main() { + std::println("magic={} big={}", get_magic(), get_big()); + return 0; +} +EOF + +# --- Part 2: #234 — a flag-vector token containing a space must reach the +# compiler as ONE argv token, not split on the space. T is injected via a +# per-glob `defines = ["T=long long"]` targeting this file specifically. If +# the space split the token, the compiler would see a bogus positional +# "long" argument and this translation unit would fail to build. +cat > src/typed_user.cpp <<'EOF' +typedef T MyLong; +long long get_big() { + MyLong big = 123456789012LL; + return static_cast(big); +} +EOF + +cat > mcpp.toml <<'EOF' +[package] +name = "incfam" +version = "0.1.0" + +[build] +cflags = ["-iquotehdr"] +flags = [ + { glob = "src/typed_user.cpp", defines = ["T=long long"] }, +] +EOF + +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "build failed"; exit 1; } + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "magic=42 big=123456789012" ]] || { + echo "unexpected output: $out"; exit 1; } + +echo "OK" diff --git a/tests/unit/test_build_flags.cpp b/tests/unit/test_build_flags.cpp index 737500ab..507b9e0a 100644 --- a/tests/unit/test_build_flags.cpp +++ b/tests/unit/test_build_flags.cpp @@ -2,6 +2,7 @@ import std; import mcpp.build.flags; +import mcpp.modgraph.scanner; namespace { @@ -91,4 +92,41 @@ TEST(BuildFlagsAtomic, StaticLinkEmittedWhenArchivePresent) { EXPECT_NE(flag.find("-latomic"), std::string::npos); } +// mcpp#226: normalize_include_flags generalizes the old -I-only +// absolutize_include_flags to the whole include/lib-search-path family, in +// BOTH the joined spelling (`-iquotehdr`) and the separated spelling +// (`-isystem` followed by a standalone next element). All four of these +// project-relative paths must resolve to the same "/proj/hdr" target. +TEST(BuildFlags, NormalizeIncludeFlagsRewritesFullIncludeFamily) { + std::filesystem::path root = "/proj"; + std::vector flags = { + "-Ihdr", "-iquotehdr", "-isystem", "hdr", "-idirafterhdr", + }; + + mcpp::modgraph::normalize_include_flags(root, flags); + + ASSERT_EQ(flags.size(), 5u); + EXPECT_EQ(flags[0], "-I" + (root / "hdr").string()); + EXPECT_EQ(flags[1], "-iquote" + (root / "hdr").string()); + EXPECT_EQ(flags[2], "-isystem"); // prefix itself untouched + EXPECT_EQ(flags[3], (root / "hdr").string()); // separated element rewritten + EXPECT_EQ(flags[4], "-idirafter" + (root / "hdr").string()); +} + +// Absolute paths and root-relative spellings are left alone (matches the +// pre-#226 -I behavior), for both the joined and separated forms. +TEST(BuildFlags, NormalizeIncludeFlagsLeavesAbsolutePathsAlone) { + std::filesystem::path root = "/proj"; + std::vector flags = { + "-I/abs/hdr", "-isystem", "/abs/hdr", "-DKEEP", + }; + + mcpp::modgraph::normalize_include_flags(root, flags); + + EXPECT_EQ(flags[0], "-I/abs/hdr"); + EXPECT_EQ(flags[1], "-isystem"); + EXPECT_EQ(flags[2], "/abs/hdr"); + EXPECT_EQ(flags[3], "-DKEEP"); +} + } // namespace diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index 2f664f07..2e72d918 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -127,6 +127,34 @@ TEST(NinjaBackend, CxxFlagsIncludeBuildIncludeDirs) { << flags.cxx; } +// Cluster A review fix (#226/#234 follow-up): `[build] include_dirs` is a +// TYPED PATH channel — bare paths from the manifest, dialect prefix applied +// at emission (-I under GNU, /I under MSVC) — not the FLAG-STRING channel +// that normalize_include_flags serves (cflags/cxxflags, where the prefix is +// already embedded in the string by the scanner). Routing dialect-prefixed +// include tokens through normalize_include_flags (whose prefix table only +// knows GNU spellings: -I/-iquote/-isystem/-idirafter/-iprefix/-L) silently +// no-ops under MSVC: "/Iinclude" matches no table entry and is never +// rewritten against plan.projectRoot, so it survives as a *relative* path — +// but ninja runs with cwd = the output dir, so the include stops resolving. +// The fix absolutizes the path directly (dialect-agnostic) before +// prepending the dialect prefix. This test would FAIL before the fix +// (emitting the literal, unrewritten "/Iinclude") and passes after. +TEST(NinjaBackend, MsvcIncludeDirsAreAbsolutizedNotGnuNormalized) { + auto plan = minimal_plan(); + plan.toolchain.compiler = mcpp::toolchain::CompilerId::MSVC; + plan.toolchain.binaryPath = "cl.exe"; + plan.toolchain.targetTriple = "x86_64-pc-windows-msvc"; + plan.manifest.buildConfig.includeDirs = {"include"}; + + auto flags = compute_flags(plan); + + auto expected = "/I" + (plan.projectRoot / "include").string(); + EXPECT_NE(flags.cxx.find(expected), std::string::npos) << flags.cxx; + // The un-rewritten, still-relative token must never appear. + EXPECT_EQ(flags.cxx.find("/Iinclude"), std::string::npos) << flags.cxx; +} + // ── assembly sources (.S/.s → asm_object via $cc, .asm → nasm_object) ──────── TEST(NinjaBackend, GasSourceUsesAsmObjectRule) { @@ -220,6 +248,48 @@ TEST(NinjaBackend, CompileCommandsSkipNasmAndCoverGas) { EXPECT_EQ(cdb.find("\"-std=c11\""), std::string::npos) << cdb; // asm-safe flags, no C std } +// mcpp#234: each packageCflags/packageCxxflags element is already one argv +// token — apply_glob_flags pushes a define like `T=long long` as the single +// element `-DT=long long`. join_flags previously joined tokens with a bare +// space and zero quoting, so once ninja resolved the command line and handed +// it to the shell, the embedded space split `-DT=long long` into TWO words +// (`-DT=long` and a bare `long`). The emitted unit_cflags line must carry the +// define as a single shell-quoted token. +TEST(NinjaBackend, QuotesFlagValueWithSpace) { + auto plan = minimal_plan(); + plan.compileUnits.push_back({ + .source = "src/main.c", + .object = "obj/main.o", + .packageName = "quote_test", + .packageCflags = {"-DT=long long"}, + }); + + auto ninja = emit_ninja_string(plan); + + EXPECT_NE(ninja.find("unit_cflags = '-DT=long long'"), std::string::npos) + << ninja; + // Must NOT appear as two bare, unquoted words split on the space. + EXPECT_EQ(ninja.find("unit_cflags = -DT=long long"), std::string::npos) + << ninja; +} + +// Plain framework-shaped flags with nothing shell-significant must pass +// through byte-for-byte unquoted (no over-quoting regression). +TEST(NinjaBackend, PlainFlagsPassThroughUnquoted) { + auto plan = minimal_plan(); + plan.compileUnits.push_back({ + .source = "src/main.cpp", + .object = "obj/main.o", + .packageName = "plain_flag_test", + .packageCxxflags = {"-DFOO=1", "-O2"}, + }); + + auto ninja = emit_ninja_string(plan); + + EXPECT_NE(ninja.find("unit_cxxflags = -DFOO=1 -O2"), std::string::npos) + << ninja; +} + TEST(NinjaBackend, RootPackageCxxflagsAreEmittedOncePerUnit) { auto plan = minimal_plan(); plan.manifest.buildConfig.cxxflags = {"-DROOT_FLAG=1"}; From 4189c82ff6429778b49909cc8c8fe10437a84a9d Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 18:22:05 +0800 Subject: [PATCH 08/21] =?UTF-8?q?docs(plan):=20C3=20(=E7=B0=87A=20#226=20#?= =?UTF-8?q?234)=20done=20=E2=80=94=20flag=20model=20+=20MSVC=20include=5Fd?= =?UTF-8?q?irs=20fix?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...8-v0.0.97-architectural-remediation-implementation-plan.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index 0b731587..833daff7 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -18,8 +18,8 @@ |---|---|---| | C1 簇E #225 | ✅ 完成 `33e6d9a` | scanner 前缀收窄+边界排除;run 复用缓存(try_fast_run);+2 单测(含确定性 glob_literal_prefix)+e2e 114;review 通过 | | C2 簇B #227 #228 | ✅ 完成 `2230872` | libs/toml AOT + expand_braces;consumer 零改(同 Value 形);+manifest 层 closed-grammar 守卫(拒非白名单 `[[x]]` typo,防静默丢数据)+brace 深度上限;+7 单测+e2e 115;review 通过 | -| C3 簇A #226 #234 | ⏳ 进行中 | | -| C4 簇C #233 #235 | ☐ 待办 | | +| C3 簇A #226 #234 | ✅ 完成 `b71d93c` | token-list 两 pass:normalize_include_flags(全 include 族 joined/separated)+shell_quote_arg;修 1 Critical=MSVC include_dirs 需先绝对化再加 dialect 前缀(typed path 通道≠flag-string 通道);+5 单测+e2e 116;review 通过 | +| C4 簇C #233 #235 | ⏳ 进行中(α) | | | C5 簇D #229 | ☐ 待办 | | | C6 簇F #232 | ☐ 待办 | | | C7 簇G #224 R6 +run-p | ☐ 待办 | | From d6b3b6cbf425f523dd75403e48e2a39a51ea4aa5 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 18:24:57 +0800 Subject: [PATCH 09/21] fix(build): evaluate cfg-conditional sources for every package via one per-package funnel (path/git deps + test mode) (#229) --- src/build/prepare.cppm | 75 ++++++++++++++++++---- tests/e2e/119_dep_cfg_sources.sh | 106 +++++++++++++++++++++++++++++++ 2 files changed, 168 insertions(+), 13 deletions(-) create mode 100755 tests/e2e/119_dep_cfg_sources.sh diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index c338771b..7c9daa99 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -337,13 +337,28 @@ materialize_generated_files(const std::filesystem::path& root, return {}; } -// L1 cfg merge for ONE manifest (root or dependency): append the matching -// conditional cflags/cxxflags/ldflags and sources (G1b) to its buildConfig. -// Sources also update the legacy modules.sources mirror — the scanner walks -// that. Conditional dependency maps are root-only and handled at the root -// call site; a dependency's conditional configs otherwise evaluate the same -// way (descriptor `target_cfg` must not be silently inert). -void merge_conditional_build(mcpp::manifest::Manifest& m, +// L1 cfg merge for ONE package's manifest (root or ANY dependency — path, +// git, or version/registry): append the matching conditional +// cflags/cxxflags/ldflags and sources (G1b) to its buildConfig. Sources also +// update the legacy modules.sources mirror — the scanner walks that. +// +// #229: this is the SINGLE funnel for cfg-conditional sources/flags — every +// package's manifest passes through exactly one call to this function, +// always immediately BEFORE that manifest is captured into `packages[]` via +// makePackageRoot()/propagateLinkFlags() (which snapshot buildConfig into +// privateBuild/linkUsage and into the root's propagated ldflags — merging +// any later than that point is silently lost for flags, though not for +// sources, which the modgraph scan re-reads live). Three call sites, one per +// loading branch, together cover every package exactly once: the root +// (before its own makePackageRoot), the path/git-dep branch, and +// loadVersionDep() (shared by the main per-dependency loop, the +// multi-version mangling secondary, and the SemVer-merge re-fetch — all three +// of ITS callers get the merge for free from the one call inside it). +// (Conditional *dependencies* are a separate, root-only concern: they must be +// merged into the dependency map BEFORE resolution even starts, so a +// dependency's own conditional deps are out of scope — see the root cfg +// block that merges `cc.dependencies` etc.) +void merge_conditional_sources_flags(mcpp::manifest::Manifest& m, const cfgpred::Ctx& ctx, std::string_view targetTriple) { @@ -784,13 +799,24 @@ prepare_build(bool print_fingerprint, } if (overrides.force_static) m->buildConfig.linkage = "static"; - // ── L1: merge platform-conditional [target.'cfg(...)'.build] flags ────── + // ── L1: merge conditional [target.'cfg(...)'.build] sources/flags AND + // root-only [target.'cfg(...)'.dependencies] ───────────────────────────── // Evaluated now (target resolved) against the resolved target — the - // --target triple for a cross build, else the host. Matching predicates' - // flags append to buildConfig, mirroring the [profile] merge above. + // --target triple for a cross build, else the host. + // + // #229: merge_conditional_sources_flags MUST run here — before + // `packages[0] = makePackageRoot(*root, *m)` snapshots `m->buildConfig` + // into `packages[0].privateBuild`/`.manifest` — because that snapshot, + // not `*m`, is what the modgraph scan and per-TU compile-flag assembly + // actually read afterward. Every dependency (path/git/version alike) gets + // the SAME treatment, at the mirror-image point in its own load path + // (right before ITS `makePackageRoot`/`propagateLinkFlags`) — see the + // dependency-manifest-acquisition block below. That makes this the root + // package's half of the one funnel, not a special case: every package is + // merged exactly once, immediately before it is captured into `packages[]`. if (!m->conditionalConfigs.empty()) { auto cc_ctx = cfgpred::context_for(overrides.target_triple); - merge_conditional_build(*m, cc_ctx, overrides.target_triple); + merge_conditional_sources_flags(*m, cc_ctx, overrides.target_triple); for (auto const& cc : m->conditionalConfigs) { if (!cfgpred::matches(cc.predicate, cc_ctx, overrides.target_triple)) continue; @@ -1710,9 +1736,18 @@ prepare_build(bool print_fingerprint, // Dependency-side L1 cfg merge (flags + sources): a descriptor's // `target_cfg` / a dep mcpp.toml's [target.'cfg(...)'.build] must - // evaluate here too — before its globs expand. + // evaluate here too — before its globs expand. This is the version/ + // registry-dep half of the #229 funnel: every loadVersionDep() caller + // (the main per-dependency loop, the multi-version mangling + // secondary, and the SemVer-merge re-fetch) shares this one call site, + // so a version dep is merged exactly once regardless of which of the + // three paths loaded it. The path/git-dep half is the mirror-image + // call right after ITS manifest load (dependency-manifest-acquisition + // block below) — same function, same one-merge-per-package guarantee, + // just keyed off a different loading branch since path/git deps never + // pass through loadVersionDep. if (!manifest->conditionalConfigs.empty()) { - merge_conditional_build(*manifest, + merge_conditional_sources_flags(*manifest, cfgpred::context_for(overrides.target_triple), overrides.target_triple); } @@ -2458,6 +2493,20 @@ prepare_build(bool print_fingerprint, name, dep_root.string(), dm.error().format())); } dep_manifest = std::move(*dm); + // #229: path/git-dep half of the L1 cfg funnel — mirrors the + // loadVersionDep call site above (loadFrom's L1 cfg merge, ~1740 + // lines up). Before this fix, path/git deps never ran this merge + // at all: their `[target.'cfg(...)'.build] sources` were parsed + // into `conditionalConfigs` but never folded into + // `buildConfig.sources` / `modules.sources`, so the modgraph scan + // never saw the file — link-time `undefined reference`. Must run + // BEFORE `propagateLinkFlags`/`makePackageRoot` below, which + // snapshot this manifest's flags/sources into `packages[]`. + if (!dep_manifest->conditionalConfigs.empty()) { + merge_conditional_sources_flags(*dep_manifest, + cfgpred::context_for(overrides.target_triple), + overrides.target_triple); + } } else { auto loaded = loadVersionDep(name, key.ns, key.shortName, spec.version); if (!loaded) return std::unexpected(loaded.error()); diff --git a/tests/e2e/119_dep_cfg_sources.sh b/tests/e2e/119_dep_cfg_sources.sh new file mode 100755 index 00000000..517c0b51 --- /dev/null +++ b/tests/e2e/119_dep_cfg_sources.sh @@ -0,0 +1,106 @@ +#!/usr/bin/env bash +# requires: gcc +# 119_dep_cfg_sources.sh — #229: a PATH DEPENDENCY's `[target.'cfg(...)'.build]` +# conditional `sources` must be compiled, in BOTH `mcpp build` and `mcpp test`. +# +# Regression: `merge_conditional_build` (the L1b cfg-sources merge) was only +# ever invoked for the root package (prepare.cppm root call site) and for +# version/registry dependencies (loadVersionDep). Path/git dependencies load +# their manifest directly and skipped the merge entirely — a path dep's +# cfg-conditional `sources` were parsed into `conditionalConfigs` but never +# folded into `buildConfig.sources` / `modules.sources`, so the modgraph scan +# never saw the file and the final link failed with `undefined reference`. +# +# `mylib` is a path dependency whose `mylib.cppm` module forward-declares +# `impl_value()` (extern "C", to sidestep name mangling) but the DEFINITION +# lives in `src/impl.cpp`, which is reachable ONLY through the cfg-conditional +# `[target.'cfg(...)'.build] sources` — never through the unconditional +# `[build] sources` glob. Host-aware: both `cfg(unix)` and `cfg(windows)` +# blocks point at the same file, so exactly one always applies (mirrors +# 86_target_cfg_dependencies.sh's host-aware dual-predicate style) and the +# test runs identically on all 3 CI platforms. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p mylib/src +cat > mylib/mcpp.toml <<'EOF' +[package] +name = "mylib" +version = "0.1.0" + +[targets.mylib] +kind = "lib" + +[build] +sources = ["src/**/*.cppm"] + +# impl.cpp is reachable ONLY through these conditional blocks — never through +# the unconditional [build] sources glob above. Exactly one of unix/windows +# always matches the resolved host target. +[target.'cfg(unix)'.build] +sources = ["src/impl.cpp"] +[target.'cfg(windows)'.build] +sources = ["src/impl.cpp"] +EOF +cat > mylib/src/mylib.cppm <<'EOF' +export module mylib; + +// Defined in src/impl.cpp, which is pulled in ONLY via cfg-conditional +// [target.'cfg(...)'.build] sources — extern "C" so the module boundary +// doesn't affect the link name. +extern "C" int impl_value(); + +export int mylib_answer() { return impl_value(); } +EOF +cat > mylib/src/impl.cpp <<'EOF' +extern "C" int impl_value() { return 42; } +EOF + +mkdir -p app/src app/tests +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +mylib = { path = "../mylib" } +EOF +cat > app/src/main.cpp <<'EOF' +import mylib; +import std; +int main() { + std::println("value = {}", mylib_answer()); + return mylib_answer() == 42 ? 0 : 1; +} +EOF +cat > app/tests/t.cpp <<'EOF' +import mylib; +int main() { + return mylib_answer() == 42 ? 0 : 1; +} +EOF + +cd app + +# (1) `mcpp build` must compile+link mylib's cfg-conditional source. +"$MCPP" build > b.log 2>&1 || { + cat b.log + echo "FAIL: build did not compile the path dep's cfg-conditional sources (undefined reference expected pre-fix)" + exit 1 +} +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "value = 42" ]] || { echo "FAIL: unexpected run output: $out"; exit 1; } + +# (2) `mcpp test` must ALSO compile+link it (the per-package funnel must fire +# in both modes, mirroring 100_feature_sources_test_mode.sh's build/test +# parity check). +"$MCPP" test > t.log 2>&1 || { + cat t.log + echo "FAIL: test mode did not compile the path dep's cfg-conditional sources" + exit 1 +} + +echo "OK" From 3b4f18e80e3c1315428fc54e812bbc83d568e7d9 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 18:53:17 +0800 Subject: [PATCH 10/21] fix(build): mirror source relative path in object paths + track compile-edge depfiles for purview/header includes (#233 #235) --- src/build/ninja_backend.cppm | 79 +++++++++++++++--- src/build/plan.cppm | 101 ++++++++++++++++------- src/build/prepare.cppm | 4 +- src/modgraph/graph.cppm | 10 +++ src/modgraph/scanner.cppm | 9 ++ tests/e2e/117_object_path_collision.sh | 76 +++++++++++++++++ tests/e2e/118_purview_include_rebuild.sh | 86 +++++++++++++++++++ tests/unit/test_ninja_backend.cpp | 34 ++++++++ 8 files changed, 359 insertions(+), 40 deletions(-) create mode 100755 tests/e2e/117_object_path_collision.sh create mode 100755 tests/e2e/118_purview_include_rebuild.sh diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index f08211ef..f114e27e 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -403,6 +403,57 @@ std::string emit_ninja_string(const BuildPlan& plan) { auto append_deps = [&] { if (msvcDeps) append(" deps = msvc\n"); }; + // mcpp#235: cxx_module/cxx_object had NO depfile at all on non-MSVC — + // only the msvcDeps branch tracked header deps (via /showIncludes). So + // editing a file #include'd inside a module's purview (or a plain + // header pulled in by a .cpp) never invalidated the compile edge: the + // P1689 scan already emits a `.dep`/`.d`-shaped list of textual + // includes, but it was generated and discarded. + // + // Naively mirroring the nasm rule below (`-MD $out.d` + `deps = gcc` + + // `depfile = $out.d`) does NOT work here: GCC's `-fmodules` bolts extra + // "reversed" rules onto ANY -M*/-MF depfile for a TU that imports or + // provides a module — e.g. for a module interface unit: + // obj/m.m.o gcm.cache/m.gcm: src/m.cppm src/vals.inc + // m.c++-module: gcm.cache/m.gcm + // .PHONY: m.c++-module + // gcm.cache/m.gcm:| obj/m.m.o + // and for an importing TU: + // obj/main.o: src/main.cpp gcm.cache/std.gcm gcm.cache/m.gcm + // obj/main.o: m.c++-module std.c++-module + // CXX_IMPORTS += m.c++-module std.c++-module + // Those extra records describe a Make-style dependency graph WITHIN the + // depfile itself (e.g. gcm.cache/m.gcm "having its own inputs"), which + // collides with gcm.cache/m.gcm already being a declared ninja-graph + // OUTPUT of this same edge (`| gcm.cache/m.gcm`) — ninja's depfile + // loader rejects that outright ("inputs may not also have inputs"), + // confirmed empirically (e2e 118 failed the fresh build this way before + // the filter below was added). GCC has no flag to suppress this. + // + // Fix: compile to a scratch `$out.d.raw`, then keep only the FIRST + // record (target + its indented continuation lines — the plain textual + // #include graph, which is all #235 needs) as `$out.d`; module BMI + // deps stay tracked independently via the existing per-edge `dyndep` + // binding, so dropping the reversed/module lines here loses nothing. + // POSIX-only (`awk`): native Windows has no POSIX shell/toolset here + // (see the existing "Windows: skip BMI restat optimization" branch + // below), so a non-MSVC Windows toolchain keeps the pre-#235 + // behavior (no depfile) rather than depend on an unavailable filter — + // msvcDeps (cl.exe) is unaffected either way (deps=msvc, no -MMD). + const bool posixDepfile = !msvcDeps && !mcpp::platform::is_windows; + const std::string mmd_flag = posixDepfile ? "-MMD -MF $out.d.raw " : ""; + const std::string mmd_filter = posixDepfile + ? " && awk 'NR==1{print;next} /^[^ ]/{exit} {print}' " + "\"$out.d.raw\" > \"$out.d\" && rm -f \"$out.d.raw\"" + : ""; + auto append_cxx_deps = [&] { + if (posixDepfile) { + append(" deps = gcc\n"); + append(" depfile = $out.d\n"); + } else { + append_deps(); + } + }; // cl.exe needs /TP (our module interfaces are .cppm, unknown to cl) and // /interface to treat the TU as a module interface unit. const std::string module_src_flags = msvcDeps ? " /interface /TP" : ""; @@ -412,19 +463,20 @@ std::string emit_ninja_string(const BuildPlan& plan) { append(std::format(" command = " "$cxx $local_includes $cxxflags $unit_cxxflags{}{} {}\n", module_output_flag, module_src_flags, compile_tail)); - append_deps(); + append_cxx_deps(); } else { append(std::format(" command = " "if [ -n \"$bmi_out\" ] && [ -f \"$bmi_out\" ]; then " "cp -p \"$bmi_out\" \"$bmi_out.bak\"; " "fi && " - "$cxx $local_includes $cxxflags $unit_cxxflags{} {} && " + "$cxx $local_includes $cxxflags $unit_cxxflags{} {}{}{} && " "if [ -n \"$bmi_out\" ] && [ -f \"$bmi_out.bak\" ] && " "cmp -s \"$bmi_out\" \"$bmi_out.bak\"; then " "mv \"$bmi_out.bak\" \"$bmi_out\"; " "else " "rm -f \"$bmi_out.bak\"; " - "fi\n", module_output_flag, compile_tail)); + "fi\n", module_output_flag, mmd_flag, compile_tail, mmd_filter)); + append_cxx_deps(); } append(" description = MOD $out\n"); if (dyndep) @@ -432,11 +484,17 @@ std::string emit_ninja_string(const BuildPlan& plan) { append("\n"); append("rule cxx_object\n"); - append(std::format( - " command = $cxx $local_includes $cxxflags $unit_cxxflags {}\n", - compile_tail)); + if constexpr (mcpp::platform::is_windows) { + append(std::format( + " command = $cxx $local_includes $cxxflags $unit_cxxflags {}\n", + compile_tail)); + } else { + append(std::format( + " command = $cxx $local_includes $cxxflags $unit_cxxflags {}{}{}\n", + mmd_flag, compile_tail, mmd_filter)); + } append(" description = OBJ $out\n"); - append_deps(); + append_cxx_deps(); if (dyndep) append(" restat = 1\n"); append("\n"); @@ -613,9 +671,10 @@ std::string emit_ninja_string(const BuildPlan& plan) { // ── Phase 1: scan edges (one .ddi per TU). ────────────────────── // .ddi is placed beside the object so multi-version mangling can // namespace by package without producing two `build` rules with - // the same `.ddi` output (plan.cppm switches `cu.object` from - // `obj/.o` to `obj//.o` whenever a basename - // collides across packages — `.ddi` follows that placement). + // the same `.ddi` output (mcpp#233: plan.cppm switches `cu.object` + // from the flat `obj/.o` to a path mirroring the source's + // relative directory under a sanitized-package prefix whenever a + // basename collides — `.ddi` follows that placement). // Skip .c files: they have no `import`s and don't need P1689 scan; // running them through cxx_scan would route them through g++ / // -fmodules which is exactly what C support is here to avoid. diff --git a/src/build/plan.cppm b/src/build/plan.cppm index c148c864..eb93a76d 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -109,17 +109,21 @@ struct BuildPlan { // choice for kind="lib" dependencies (see plan.cppm). bool source_defines_main(const std::filesystem::path& src); -// Build a BuildPlan from already-validated inputs. -BuildPlan make_plan(const mcpp::manifest::Manifest& manifest, - const mcpp::toolchain::Toolchain& tc, - const mcpp::toolchain::Fingerprint& fp, - const mcpp::modgraph::Graph& graph, - const std::vector& topoOrder, - const std::vector& packages, - const std::filesystem::path& projectRoot, - const std::filesystem::path& outputDir, - const std::filesystem::path& stdBmiPath, - const std::filesystem::path& stdObjectPath); +// Build a BuildPlan from already-validated inputs. Fails (mcpp#233) only +// when the object-path uniqueness assertion below finds a residual +// collision after the relPath-mirroring scheme — a would-be ninja +// "multiple rules generate X" turned into a diagnosable mcpp error. +std::expected +make_plan(const mcpp::manifest::Manifest& manifest, + const mcpp::toolchain::Toolchain& tc, + const mcpp::toolchain::Fingerprint& fp, + const mcpp::modgraph::Graph& graph, + const std::vector& topoOrder, + const std::vector& packages, + const std::filesystem::path& projectRoot, + const std::filesystem::path& outputDir, + const std::filesystem::path& stdBmiPath, + const std::filesystem::path& stdObjectPath); } // namespace mcpp::build @@ -320,16 +324,17 @@ bool source_defines_main(const std::filesystem::path& src) { return false; } -BuildPlan make_plan(const mcpp::manifest::Manifest& manifest, - const mcpp::toolchain::Toolchain& tc, - const mcpp::toolchain::Fingerprint& fp, - const mcpp::modgraph::Graph& graph, - const std::vector& topoOrder, - const std::vector& packages, - const std::filesystem::path& projectRoot, - const std::filesystem::path& outputDir, - const std::filesystem::path& stdBmiPath, - const std::filesystem::path& stdObjectPath) +std::expected +make_plan(const mcpp::manifest::Manifest& manifest, + const mcpp::toolchain::Toolchain& tc, + const mcpp::toolchain::Fingerprint& fp, + const mcpp::modgraph::Graph& graph, + const std::vector& topoOrder, + const std::vector& packages, + const std::filesystem::path& projectRoot, + const std::filesystem::path& outputDir, + const std::filesystem::path& stdBmiPath, + const std::filesystem::path& stdObjectPath) { BuildPlan plan; plan.manifest = manifest; @@ -425,8 +430,21 @@ BuildPlan make_plan(const mcpp::manifest::Manifest& manifest, // 1a. Detect basename collisions (both cross-package AND intra-package: // ftxui ships dom/color.cpp + screen/color.cpp, for instance). - // For colliding files the object path gets a per-unit prefix - // derived from `/` so collisions are impossible. + // For colliding files the object path gets a per-unit prefix. + // + // mcpp#233: the prefix used to be derived from just the file's + // IMMEDIATE parent directory name (`_`), which + // itself collides whenever two files share a parent dir NAME at + // different depths — e.g. a/src/util.cpp and b/src/util.cpp both + // fold to `_src/util.o`, and ninja rejects the plan with + // "multiple rules generate obj/...". The prefix now mirrors the + // unit's FULL relative directory instead (SourceUnit::relPath, set + // by the scanner against the unit's own package root), which is + // unique by construction: two distinct files under one package + // root can never share both relPath and basename. Non-colliding + // files keep the pre-existing flat `obj/` layout untouched + // (back-compat for the overwhelmingly common single-file-per- + // basename project). std::map basenameCount; for (auto idx : topoOrder) { basenameCount[object_filename_for(graph.units[idx].path, objExt)]++; @@ -449,13 +467,10 @@ BuildPlan make_plan(const mcpp::manifest::Manifest& manifest, cu.packageAsmflags = u.packageAsmflags; const auto fname = object_filename_for(u.path, objExt); if (basenameCount[fname] > 1) { - // Use / as prefix to handle - // both cross-package (multi-version mangling) and intra-package - // (e.g. ftxui dom/color.cpp vs screen/color.cpp) collisions. - auto parentDir = u.path.parent_path().filename().string(); + auto relDir = u.relPath.parent_path(); auto prefix = u.packageName.empty() - ? parentDir - : sanitize(u.packageName) + "_" + parentDir; + ? relDir + : std::filesystem::path(sanitize(u.packageName)) / relDir; cu.object = std::filesystem::path("obj") / prefix / fname; } else { cu.object = std::filesystem::path("obj") / fname; @@ -468,6 +483,34 @@ BuildPlan make_plan(const mcpp::manifest::Manifest& manifest, plan.compileUnits.push_back(std::move(cu)); } + // 1b. mcpp#233: uniqueness assertion. The relDir-mirroring prefix above + // is unique by construction for any two distinct source files (see the + // reasoning in 1a), so this should never fire — it is a defensive + // backstop turning a would-be ninja "multiple rules generate X" hard + // stop into a diagnosable mcpp error that names the colliding sources, + // for any input the reasoning above didn't anticipate. + { + std::map> byObject; + for (auto& cu : plan.compileUnits) { + byObject[cu.object.generic_string()].push_back(cu.source); + } + std::string collisions; + for (auto& [obj, srcs] : byObject) { + if (srcs.size() < 2) continue; + if (!collisions.empty()) collisions += "; "; + collisions += obj + " <- "; + for (std::size_t i = 0; i < srcs.size(); ++i) { + if (i) collisions += ", "; + collisions += srcs[i].string(); + } + } + if (!collisions.empty()) { + return std::unexpected(std::format( + "internal error: object path collision after uniqueness " + "pass (please report): {}", collisions)); + } + } + // 2. Build map of module-name → compile unit (for inter-unit dep resolution) std::map producerOf; for (std::size_t i = 0; i < plan.compileUnits.size(); ++i) { diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index c338771b..dd32f75c 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -2981,9 +2981,11 @@ prepare_build(bool print_fingerprint, ctx.outputDir = target_dir(*tc, fp, *root); ctx.stdBmi = stdBmiPath; ctx.stdObject = stdObjectPath; - ctx.plan = mcpp::build::make_plan(*m, *tc, fp, scan.graph, report.topoOrder, + auto planResult = mcpp::build::make_plan(*m, *tc, fp, scan.graph, report.topoOrder, packages, *root, ctx.outputDir, stdBmiPath, stdObjectPath); + if (!planResult) return std::unexpected(planResult.error()); + ctx.plan = std::move(*planResult); ctx.plan.stdCompatBmiPath = stdCompatBmiPath; ctx.plan.stdCompatObjectPath = stdCompatObjectPath; diff --git a/src/modgraph/graph.cppm b/src/modgraph/graph.cppm index 20ebf5cc..38878ca8 100644 --- a/src/modgraph/graph.cppm +++ b/src/modgraph/graph.cppm @@ -14,6 +14,16 @@ struct ModuleId { struct SourceUnit { std::filesystem::path path; + // mcpp#233: path relative to this unit's PACKAGE ROOT (not the primary + // project root — a path dependency has its own root), set by the + // scanner where the root is known. Used by plan.cppm to mirror the + // source's directory layout into its object path so that two + // same-named files under different subdirectories (e.g. a/src/util.cpp + // and b/src/util.cpp) never fold onto the same object output. Left + // default-constructed (empty) for units synthesized outside the + // scanner (e.g. plan.cppm's ad-hoc main.cpp CompileUnit, which never + // reads this field). + std::filesystem::path relPath; std::string packageName; std::vector localIncludeDirs; std::vector packageCflags; diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 8703d39d..10e9126c 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -852,6 +852,10 @@ void scan_one_into(ScanResult& result, if (ov) { SourceUnit u; u.path = f; + // mcpp#233: relative to the PACKAGE root (`root`), not the + // primary project root — a path dependency scans with its own + // root here. + u.relPath = std::filesystem::relative(f, root); u.packageName = qualifiedName; u.scanOverridden = true; if (!ov->provides.empty()) { @@ -881,6 +885,9 @@ void scan_one_into(ScanResult& result, result.errors.push_back(r.error()); continue; } + // mcpp#233: relative to the PACKAGE root (`root`), matching the + // scan_overrides branch above. + r->relPath = std::filesystem::relative(f, root); r->localIncludeDirs = localIncludeDirs; r->packageCflags = packageCflags; r->packageCxxflags = packageCxxflags; @@ -994,6 +1001,8 @@ ScanResult scan_packages_p1689(const std::vector& packages, result.errors.push_back(ScanError{ f, 0, r.error() }); continue; } + // mcpp#233: same relPath contract as scan_one_into above. + r->relPath = std::filesystem::relative(f, p.root); r->localIncludeDirs = localIncludeDirs; r->packageCflags = p.usageResolved ? p.privateBuild.cflags diff --git a/tests/e2e/117_object_path_collision.sh b/tests/e2e/117_object_path_collision.sh new file mode 100755 index 00000000..e1aad4f7 --- /dev/null +++ b/tests/e2e/117_object_path_collision.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# requires: +# mcpp#233: object paths must mirror the source's relative directory, not +# fold onto `obj/_/`. Two sources with +# the SAME basename under DIFFERENT subtrees that happen to share the same +# immediate parent directory NAME — a/src/util.cpp and b/src/util.cpp, both +# under a literal `src/` — used to fold onto the same object output +# (`obj/_src/util.o`), and ninja rejected the generated plan with +# "multiple rules generate obj/...". This must now build successfully, with +# each source compiling to its own, distinct object. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +"$MCPP" new objcollide > /dev/null +cd objcollide + +mkdir -p a/src b/src + +cat > a/src/util.cpp <<'EOF' +int fa() { return 1; } +EOF + +cat > b/src/util.cpp <<'EOF' +int fb() { return 2; } +EOF + +cat > src/main.cpp <<'EOF' +import std; +extern int fa(); +extern int fb(); +int main() { + int sum = fa() + fb(); + std::println("sum={}", sum); + return sum == 3 ? 0 : 1; +} +EOF + +cat > mcpp.toml <<'EOF' +[package] +name = "objcollide" +version = "0.1.0" +[modules] +sources = ["src/**/*.cppm", "src/**/*.cpp", "a/src/**/*.cpp", "b/src/**/*.cpp"] +[targets.objcollide] +kind = "bin" +main = "src/main.cpp" +EOF + +"$MCPP" build > build.log 2>&1 || { + cat build.log + echo "FAIL: build failed (expected: colliding basenames must not collapse to the same object path)" + exit 1 +} + +ninja_file="$(find target -name build.ninja | head -1)" +[[ -n "$ninja_file" ]] || { echo "no build.ninja generated"; exit 1; } + +# Both util.cpp compile edges must be present, each with a DISTINCT object +# output (mirroring a/src vs b/src rather than folding both onto the same +# "..._src/util.o"). +util_objects="$(grep -oE 'build [^ ]*util[^ ]*\.o : cxx_object' "$ninja_file" | awk '{print $2}' | sort -u)" +util_count="$(echo "$util_objects" | grep -c . || true)" +[[ "$util_count" == "2" ]] || { + echo "FAIL: expected 2 distinct util.cpp object outputs, got $util_count:" + echo "$util_objects" + cat "$ninja_file" + exit 1 +} + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "sum=3" ]] || { echo "unexpected output: $out"; exit 1; } + +echo "OK" diff --git a/tests/e2e/118_purview_include_rebuild.sh b/tests/e2e/118_purview_include_rebuild.sh new file mode 100755 index 00000000..ff0eccbe --- /dev/null +++ b/tests/e2e/118_purview_include_rebuild.sh @@ -0,0 +1,86 @@ +#!/usr/bin/env bash +# requires: +# mcpp#235: compile edges must track header/purview/GMF `#include`s via a +# depfile. Before this fix, `cxx_module`/`cxx_object` had NO depfile on +# non-MSVC (only the msvcDeps branch added `deps=msvc`) — the P1689 scan's +# `$out.dep` was generated then discarded. So editing a file `#include`d +# inside a module's purview (or a plain header included by a .cpp) did NOT +# invalidate the compile edge: `mcpp run` kept printing stale output. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +"$MCPP" new purviewinc > /dev/null +cd purviewinc + +cat > src/vals.inc <<'EOF' +export inline int answer() { return 41; } +EOF + +cat > src/m.cppm <<'EOF' +export module m; +#include "vals.inc" +EOF + +cat > src/main.cpp <<'EOF' +import std; +import m; +int main() { + std::println("{}", answer()); + return 0; +} +EOF + +cat > mcpp.toml <<'EOF' +[package] +name = "purviewinc" +version = "0.1.0" +EOF + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "41" ]] || { echo "unexpected initial output: $out"; exit 1; } + +sed -i 's/41/42/' src/vals.inc + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "42" ]] || { + echo "FAIL: editing a purview #include'd file did not trigger a rebuild" + echo "got: $out (expected 42)" + exit 1 +} + +# --- Second assertion: a plain .h included by a .cpp must also rebuild ---- +cat > src/helper.h <<'EOF' +inline int helper_val() { return 100; } +EOF + +cat > src/helper_user.cpp <<'EOF' +#include "helper.h" +int use_helper() { return helper_val(); } +EOF + +cat > src/main.cpp <<'EOF' +import std; +import m; +extern int use_helper(); +int main() { + std::println("{} {}", answer(), use_helper()); + return 0; +} +EOF + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "42 100" ]] || { echo "unexpected output before header edit: $out"; exit 1; } + +sed -i 's/100/200/' src/helper.h + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "42 200" ]] || { + echo "FAIL: editing a plain .h included by a .cpp did not trigger a rebuild" + echo "got: $out (expected 42 200)" + exit 1 +} + +echo "OK" diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index 2e72d918..2267be2b 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -71,6 +71,40 @@ TEST(NinjaBackend, ObjectiveCSourceUsesCObjectRuleAndCFlags) { << ninja; } +// mcpp#235: cxx_module/cxx_object must track header/purview/GMF includes via +// a GNU-style depfile on non-MSVC toolchains (this test's plan uses GCC on a +// non-Windows host, so posixDepfile is true). Before the fix, neither rule +// had ANY depfile outside the msvcDeps branch, so editing a file #include'd +// inside a module's purview (or a header pulled in by a .cpp) never +// invalidated the compile edge. The depfile is routed through a scratch +// `$out.d.raw` + `awk` filter (not written directly to `$out.d`) because +// GCC's `-fmodules` bolts non-standard "reversed" module rules onto the raw +// -MMD output that ninja's depfile loader rejects — see the long comment at +// the definition site for the empirically-confirmed failure mode. +TEST(NinjaBackend, CxxModuleAndCxxObjectRulesTrackHeaderDepsViaGccDepfile) { + auto plan = minimal_plan(); + + auto ninja = emit_ninja_string(plan); + + auto module_rule_start = ninja.find("rule cxx_module"); + auto object_rule_start = ninja.find("rule cxx_object"); + ASSERT_NE(module_rule_start, std::string::npos) << ninja; + ASSERT_NE(object_rule_start, std::string::npos) << ninja; + ASSERT_LT(module_rule_start, object_rule_start) << ninja; + + auto module_rule = ninja.substr(module_rule_start, object_rule_start - module_rule_start); + auto object_rule = ninja.substr(object_rule_start); + + for (auto const& rule : {module_rule, object_rule}) { + EXPECT_NE(rule.find("-MMD -MF $out.d.raw"), std::string::npos) << ninja; + EXPECT_NE(rule.find("deps = gcc"), std::string::npos) << ninja; + EXPECT_NE(rule.find("depfile = $out.d\n"), std::string::npos) << ninja; + // The raw compiler depfile (with GCC's module-specific reversed + // rules) must never be bound directly as ninja's depfile. + EXPECT_EQ(rule.find("depfile = $out.d.raw"), std::string::npos) << ninja; + } +} + TEST(NinjaBackend, UsesPackageCppStandardForCxxFlags) { auto plan = minimal_plan(); plan.manifest.package.standard = "c++26"; From f5ed2e0a19416903b59f757b4966f7b9c89bc0cd Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 18:56:01 +0800 Subject: [PATCH 11/21] =?UTF-8?q?docs(plan):=20C4=20(=E7=B0=87C)=20done=20?= =?UTF-8?q?on=20=CE=B1=20+=20C5=20(=E7=B0=87D)=20done=20on=20=CE=B2?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...v0.0.97-architectural-remediation-implementation-plan.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index 833daff7..1b043555 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -19,9 +19,9 @@ | C1 簇E #225 | ✅ 完成 `33e6d9a` | scanner 前缀收窄+边界排除;run 复用缓存(try_fast_run);+2 单测(含确定性 glob_literal_prefix)+e2e 114;review 通过 | | C2 簇B #227 #228 | ✅ 完成 `2230872` | libs/toml AOT + expand_braces;consumer 零改(同 Value 形);+manifest 层 closed-grammar 守卫(拒非白名单 `[[x]]` typo,防静默丢数据)+brace 深度上限;+7 单测+e2e 115;review 通过 | | C3 簇A #226 #234 | ✅ 完成 `b71d93c` | token-list 两 pass:normalize_include_flags(全 include 族 joined/separated)+shell_quote_arg;修 1 Critical=MSVC include_dirs 需先绝对化再加 dialect 前缀(typed path 通道≠flag-string 通道);+5 单测+e2e 116;review 通过 | -| C4 簇C #233 #235 | ⏳ 进行中(α) | | -| C5 簇D #229 | ☐ 待办 | | -| C6 簇F #232 | ☐ 待办 | | +| C4 簇C #233 #235 | ✅ 完成 `3b4f18e`(α) | 对象路径:保留 basename 碰撞门,消歧前缀改 relPath.parent_path()(非碰撞项字节不变,4 个断言扁平路径 e2e 保绿)+唯一性断言;depfile:cxx_module/object 加 -MMD,过滤 GCC -fmodules 反向规则(awk 保留文本 include 记录,POSIX 门控);+2 e2e 117/118+1 单测;review 通过 | +| C5 簇D #229 | ✅ 完成 `d6b3b6c`(β) | 3-site 漏斗(root+version-dep 保留,新增 path/git-dep site=真正 #229 修复);偏离 brief 有据(apply()-loop 会丢 cfg flags:makePackageRoot 早快照 privateBuild);+e2e 119;review 通过(3 风险全清) | +| C6 簇F #232 | ⏳ 进行中(β) | | | C7 簇G #224 R6 +run-p | ☐ 待办 | | | C8 版本 0.0.97 | ☐ 待办 | | | PR / CI / 合入 / release | ☐ 待办 | | From 482b428d333e60b07fe42b14383325e4df535f02 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 19:14:55 +0800 Subject: [PATCH 12/21] fix(nasm): provision nasm through the toolchain's synchronous resolve_xpkg_path gate (index refresh + hard error); stop swallowing config bootstrap errors (#232) --- src/build/prepare.cppm | 32 +++++- src/xlings.cppm | 42 +++---- tests/e2e/105_asm_sources_nasm.sh | 184 +++++++++++++++++++++++++++++- tests/unit/test_xlings.cpp | 162 ++++++++++++++++++++++++++ 4 files changed, 394 insertions(+), 26 deletions(-) diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 7c9daa99..44c73b24 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -3073,11 +3073,35 @@ prepare_build(bool print_fingerprint, "`!`-exclude glob in [build].sources)", trip.str())); } ctx.plan.nasmFormat = *fmt; + + // #232: nasm used to go through a bespoke `ensure_nasm` path + // whose `if (cfgNasm)` guard silently swallowed a `get_cfg()` + // bootstrap failure (misreporting it as "no nasm"), and whose + // install fallback never refreshed the package index and + // downgraded a failed install to a warning. Surface the real + // config error, then provision through the SAME synchronous + // gate the compiler toolchain uses (index refresh before + // install, blocking install, hard error on failure) — see the + // toolchain resolution block above (~line 872-899). auto cfgNasm = get_cfg(); - std::optional nasmBin; - if (cfgNasm) { - nasmBin = mcpp::xlings::ensure_nasm( - mcpp::config::make_xlings_env(**cfgNasm), /*quiet=*/false, {}); + if (!cfgNasm) return std::unexpected(cfgNasm.error()); + + std::optional nasmBin = + mcpp::xlings::find_usable_nasm(mcpp::config::make_xlings_env(**cfgNasm)); + if (!nasmBin) { + mcpp::fetcher::Fetcher nasmFetcher(**cfgNasm); + mcpp::fetcher::InstallProgressHandler nasmProgress; + auto nasmTarget = std::format("xim:nasm@{}", + mcpp::xlings::pinned::kNasmVersion); + auto payload = nasmFetcher.resolve_xpkg_path( + nasmTarget, /*autoInstall=*/true, &nasmProgress); + if (!payload) { + return std::unexpected(std::format( + "NASM sources (.asm) present but nasm provisioning " + "failed: {}", payload.error().message)); + } + nasmBin = mcpp::xlings::find_sandbox_nasm( + mcpp::config::make_xlings_env(**cfgNasm)); } if (!nasmBin) { return std::unexpected(std::string( diff --git a/src/xlings.cppm b/src/xlings.cppm index 358d6383..92b756a1 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -34,6 +34,7 @@ namespace pinned { inline constexpr std::string_view kPatchelfVersion = "0.18.0"; inline constexpr std::string_view kNinjaVersion = "1.12.1"; inline constexpr std::string_view kXlingsVersion = "0.4.51"; + inline constexpr std::string_view kNasmVersion = "3.02"; } // ─── Path helpers (pure functions, no subprocess) ─────────────────── @@ -266,12 +267,23 @@ void ensure_patchelf(const Env& env, bool quiet, void ensure_ninja(const Env& env, bool quiet, const BootstrapProgressCallback& cb); -// Resolve a usable nasm (≥ kNasmMinMajor.kNasmMinMinor): PATH first (CI -// images ship one), the mcpp sandbox second, `xlings install nasm` third. -// Called lazily — only when a build plan actually contains .asm units. The -// caller HARD-FAILS on nullopt; assembly sources are never silently skipped. -std::optional ensure_nasm(const Env& env, bool quiet, - const BootstrapProgressCallback& cb); +// Fast, side-effect-free probe for a usable nasm (>= 2.16): PATH first (CI +// images ship one), the mcpp sandbox second. NEVER triggers an install — +// pure lookup. A build that needs to *provision* nasm goes through the +// toolchain's synchronous fetcher gate +// (mcpp::pm::Fetcher::resolve_xpkg_path("xim:nasm@", autoInstall, +// ...)) from mcpp.build.prepare, the same gate the compiler toolchain uses: +// this module is a LEAF dependency and must not import mcpp.config / +// mcpp.fetcher (#232 — the old bespoke `ensure_nasm` install path skipped +// the index refresh and downgraded install failure to a warning). +std::optional find_usable_nasm(const Env& env); + +// Locate an already-installed nasm inside the mcpp sandbox +// ($XLINGS_HOME/data/xpkgs/xim-x-nasm//...). Pure lookup: no PATH +// probe, no install. Called lazily — only when a build plan actually +// contains .asm units — after a provisioning gate has landed the package; +// the caller HARD-FAILS on nullopt, never silently skips assembly sources. +std::optional find_sandbox_nasm(const Env& env); // ─── Index freshness ──────────────────────────────────────────────── @@ -1257,6 +1269,8 @@ bool nasm_version_ok(const std::filesystem::path& bin) { return major > 2 || (major == 2 && minor >= 16); } +} // namespace + std::optional find_sandbox_nasm(const Env& env) { auto root = paths::xim_tool_root(env, "nasm"); auto nasm_name = std::string("nasm") + std::string(mcpp::platform::exe_suffix); @@ -1271,26 +1285,12 @@ std::optional find_sandbox_nasm(const Env& env) { return std::nullopt; } -} // namespace - -std::optional ensure_nasm(const Env& env, bool quiet, - const BootstrapProgressCallback& cb) -{ +std::optional find_usable_nasm(const Env& env) { auto nasm_name = std::string("nasm") + std::string(mcpp::platform::exe_suffix); if (auto sys = mcpp::platform::fs::which(nasm_name); sys && nasm_version_ok(*sys)) { return sys; } - if (auto boxed = find_sandbox_nasm(env)) return boxed; - - if (!quiet) - print_status("Bootstrap", "nasm into mcpp sandbox (one-time)"); - mcpp::log::ScopedTimer _t_nasm("init", "bootstrap nasm"); - int rc = install_with_progress(env, "xim:nasm", cb, quiet); - if (rc != 0 && !quiet) { - std::println(stderr, - "warning: failed to bootstrap nasm into mcpp sandbox (exit {})", rc); - } return find_sandbox_nasm(env); } diff --git a/tests/e2e/105_asm_sources_nasm.sh b/tests/e2e/105_asm_sources_nasm.sh index 72d1d2ee..d8ddbd6d 100755 --- a/tests/e2e/105_asm_sources_nasm.sh +++ b/tests/e2e/105_asm_sources_nasm.sh @@ -8,8 +8,12 @@ set -e [[ "$(uname -m)" == "x86_64" ]] || { echo "SKIP: x86-only NASM"; exit 0; } +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" + TMP=$(mktemp -d) -trap "rm -rf $TMP" EXIT +COLD=$(mktemp -d) +WARM=$(mktemp -d) +trap "rm -rf '$TMP' '$COLD' '$WARM'" EXIT cd "$TMP" "$MCPP" new nasmix > /dev/null @@ -79,4 +83,182 @@ out="$("$MCPP" run 2>&1 | tail -1)" [[ "$out" == "asm mul2 = 26" ]] || { echo "stale after %include change: $out (depfile tracking broken)"; exit 1; } +echo "OK (warm PATH nasm)" + +# ─── #232 cold-env variant A: config-bootstrap error is not swallowed ── +# +# Root cause (issue #232): nasm used a bespoke, weaker provisioning path. +# An `if (cfgNasm)` guard silently swallowed a `get_cfg()` bootstrap +# failure, misreporting it as the generic "no usable nasm" message instead +# of the real cause. The fix surfaces `get_cfg()`'s error directly. +# +# To isolate THIS guard (as opposed to the toolchain-resolution gate, +# which already correctly hard-errors on a broken config), toolchain +# resolution must not itself need `get_cfg()`: `[toolchain] linux = +# "system"` is the escape hatch that resolves the compiler straight off +# $CXX/PATH, with no xlings/config involvement. A project that doesn't +# `import std;` also skips the (unrelated) std-module precompile cache, +# whose own cache-dir resolution is independent of this gate. That way +# the FIRST config-bootstrap failure the build ever surfaces is the one +# inside the nasm gate itself — deterministic, no network required. +HOSTCXX="" +for c in /usr/bin/clang++ /usr/bin/g++ clang++ g++ c++; do + p="$(command -v "$c" 2>/dev/null)" && { HOSTCXX="$p"; break; } +done + +if [[ -z "$HOSTCXX" ]]; then + echo "SKIP: no host C++ compiler found for the cold-env (#232) variant" +else + cd "$COLD" + "$MCPP" new nasmix-cold > /dev/null + cd nasmix-cold + + mkdir -p inc + cat > inc/consts.inc <<'EOF' +%define MUL_BIAS 5 +EOF + cat > src/mul2.asm <<'EOF' +%include "consts.inc" +section .text +global asm_mul2 +asm_mul2: + lea rax, [rdi*2 + MUL_BIAS] + ret +EOF + cat > src/main.cpp <<'EOF' +#include +extern "C" long asm_mul2(long a); +int main() { + printf("asm mul2 = %ld\n", asm_mul2(10)); + return asm_mul2(10) == 25 ? 0 : 1; +} +EOF + cat > mcpp.toml <<'EOF' +[package] +name = "nasmix-cold" +version = "0.1.0" + +[toolchain] +linux = "system" + +[build] +include_dirs = ["inc"] +EOF + + # A regular FILE where $MCPP_HOME must be a directory: config + # bootstrap's directory-creation loop fails deterministically, no + # network involved. + : > "$COLD/brokenhome" + + if out=$(MCPP_HOME="$COLD/brokenhome" CXX="$HOSTCXX" "$MCPP" build 2>&1); then + echo "cold-env (#232) build unexpectedly succeeded:" + echo "$out" + exit 1 + fi + echo "$out" | grep -qi "cannot create" || { + echo "cold-env build failed, but not with the real bootstrap error:" + echo "$out"; exit 1; } + if echo "$out" | grep -qi "no usable nasm"; then + echo "REGRESSION (#232): the guard swallowed the config-bootstrap" + echo "error and fell back to the generic 'no usable nasm' message:" + echo "$out" + exit 1 + fi + + echo "OK (cold-env: real bootstrap error surfaced, not swallowed)" +fi + +# ─── #232 cold-env variant B: nasm absent everywhere routes through the ── +# ─── synchronous fetcher gate (never the retired bespoke message) ─────── +# +# nasm hidden from both PATH (decoy, too-old version) and the sandbox +# forces the provisioning branch: `Fetcher::resolve_xpkg_path("xim:nasm@…", +# autoInstall=true, …)` — the SAME gate the compiler toolchain uses. This +# sandbox has network, but package-index/mirror conditions can be flaky in +# CI (rate limiting, mirror selection), so the assertion is deliberately +# robust to either outcome: a successful synchronous install (the common +# case) OR a real, specific failure. What #232 guarantees either way is +# that the OLD bespoke path's generic "no usable nasm ... was found or +# installable" message (which fires even when the real cause is a broken +# index/network) never appears again. +export MCPP_HOME="$WARM/home" +mkdir -p "$MCPP_HOME" +source "$SCRIPT_DIR/_inherit_toolchain.sh" +# Force a cold nasm even if the host machine already has it installed. +rm -rf "$MCPP_HOME/registry/data/xpkgs/xim-x-nasm" + +DECOY="$WARM/decoy" +mkdir -p "$DECOY" +cat > "$DECOY/nasm" <<'EOF' +#!/usr/bin/env bash +# Deliberately too old (< 2.16): the version gate must reject it so the +# PATH probe falls through to sandbox lookup / provisioning, never a +# false "found". +[[ "$1" == "-v" ]] && echo "NASM version 2.10.00 compiled on Jan 1 2020" +exit 0 +EOF +chmod +x "$DECOY/nasm" +export PATH="$DECOY:$PATH" + +mkdir -p "$WARM/proj" +cd "$WARM/proj" +"$MCPP" new nasmix-warm > /dev/null +cd nasmix-warm + +mkdir -p inc +cat > inc/consts.inc <<'EOF' +%define MUL_BIAS 5 +EOF +cat > src/mul2.asm <<'EOF' +%include "consts.inc" +section .text +global asm_mul2 +asm_mul2: + lea rax, [rdi*2 + MUL_BIAS] + ret +EOF +cat > src/main.cpp <<'EOF' +import std; +extern "C" long asm_mul2(long a); +int main() { + std::println("asm mul2 = {}", asm_mul2(10)); + return asm_mul2(10) == 25 ? 0 : 1; +} +EOF +cat > mcpp.toml <<'EOF' +[package] +name = "nasmix-warm" +version = "0.1.0" + +[build] +include_dirs = ["inc"] +EOF + +if out=$("$MCPP" build 2>&1); then + # Synchronous provisioning succeeded: nasm must have actually landed + # in the sandbox (proves the fetcher gate — not the decoy — supplied + # the binary) and the program must run correctly. + find "$MCPP_HOME/registry/data/xpkgs/xim-x-nasm" -type f -name 'nasm*' 2>/dev/null | grep -q . || { + echo "build succeeded but nasm never landed in the sandbox (#232 gate not exercised):" + echo "$out"; exit 1; } + run_out="$("$MCPP" run 2>&1 | tail -1)" + [[ "$run_out" == "asm mul2 = 25" ]] || { + echo "cold-provisioned build produced wrong output: $run_out"; exit 1; } + echo "OK (cold-env: nasm synchronously provisioned via the fetcher gate)" +else + if echo "$out" | grep -qi "no usable nasm"; then + echo "REGRESSION (#232): fell back to the old generic 'no usable" + echo "nasm' message instead of the real provisioning failure:" + echo "$out" + exit 1 + fi + echo "$out" | grep -Eqi "provisioning failed|xlings install" || { + echo "cold-env build failed without a real provisioning error message:" + echo "$out"; exit 1; } + echo "NOTE: nasm cold-provisioning hit an environment/network condition" + echo "in this sandbox (not a #232 regression) — the real error surfaced" + echo "correctly instead of the old generic message:" + echo "$out" | grep -i "provisioning failed" +fi + echo "OK" diff --git a/tests/unit/test_xlings.cpp b/tests/unit/test_xlings.cpp index 9187f461..ccbab5f3 100644 --- a/tests/unit/test_xlings.cpp +++ b/tests/unit/test_xlings.cpp @@ -15,6 +15,168 @@ std::filesystem::path make_tempdir(std::string_view name) { } // namespace +// ─── find_usable_nasm / find_sandbox_nasm (issue #232) ──────────────── +// +// #232: nasm used to go through a bespoke `ensure_nasm` that could trigger +// a side-effecting install of its own. The fix splits this into a pure, +// side-effect-free PATH+sandbox probe (`find_usable_nasm` / +// `find_sandbox_nasm`, tested here) and a SEPARATE synchronous +// provisioning step in mcpp.build.prepare that goes through the same +// `Fetcher::resolve_xpkg_path` gate the compiler toolchain uses (that step +// needs `mcpp.config`/`mcpp.fetcher`, which this LEAF module — mcpp.xlings +// — cannot import, so it isn't unit-testable from here without network; +// see tests/e2e/105_asm_sources_nasm.sh for the end-to-end coverage). + +#if !defined(_WIN32) + +namespace { + +// A fake `nasm` shell script reporting a fixed `-v` version string, laid +// out at /nasm so it can be resolved either via PATH `which()` or via +// the sandbox layout (//nasm or ...//bin/nasm). +std::filesystem::path make_fake_nasm(const std::filesystem::path& dir, + std::string_view versionLine) { + std::filesystem::create_directories(dir); + auto nasm = dir / "nasm"; + std::ofstream os(nasm); + // `#!/bin/sh` (a single exec) rather than `#!/usr/bin/env bash` (an + // extra `env` -> `bash` hop) — one less fork/exec generation for the + // nested `mcpp test` -> test-binary -> capture_exec chain to cross. + os << "#!/bin/sh\n" + << "if [ \"$1\" = \"-v\" ]; then\n" + << " echo \"" << versionLine << "\"\n" + << " exit 0\n" + << "fi\n" + << "exit 1\n"; + os.close(); + std::filesystem::permissions( + nasm, + std::filesystem::perms::owner_exec + | std::filesystem::perms::owner_read + | std::filesystem::perms::owner_write); + return nasm; +} + +// The current PATH, minus any directory that already has a real `nasm` on +// it. `find_usable_nasm`'s version check shells out (`env`/`bash` must +// still be resolvable), so tests must NOT collapse PATH down to a single +// directory — only remove the entries that would shadow the fake binary +// under test. +std::string path_without_real_nasm() { + auto path = mcpp::platform::env::get("PATH").value_or(""); + std::vector kept; + std::size_t start = 0; + while (start <= path.size()) { + auto end = path.find(':', start); + auto dir = path.substr(start, end == std::string::npos ? std::string::npos : end - start); + if (!dir.empty() && !std::filesystem::exists(std::filesystem::path(dir) / "nasm")) { + kept.push_back(dir); + } + if (end == std::string::npos) break; + start = end + 1; + } + std::string out; + for (auto& d : kept) { + if (!out.empty()) out += ':'; + out += d; + } + return out; +} + +} // namespace + +TEST(FindUsableNasm, FindsOnPathWhenVersionIsAdequate) { + auto pathDir = make_tempdir("mcpp-nasm-path"); + auto expected = make_fake_nasm(pathDir, "NASM version 2.16.03 compiled on Jan 1 2026"); + + // Prepend (not replace) PATH: the fake binary must shadow any real + // system nasm ahead of it, but `env`/`bash` must stay resolvable for + // the fake script's own shebang + the version-check subprocess. + auto base = path_without_real_nasm(); + mcpp::platform::env::ScopedEnv path("PATH", pathDir.string() + ":" + base); + + auto home = make_tempdir("mcpp-nasm-home"); // no sandbox nasm present + mcpp::xlings::Env env{.home = home}; + + auto found = mcpp::xlings::find_usable_nasm(env); + ASSERT_TRUE(found.has_value()); + EXPECT_EQ(*found, expected); + + std::filesystem::remove_all(pathDir); + std::filesystem::remove_all(home); +} + +TEST(FindUsableNasm, RejectsPathNasmBelowMinVersionAndFindsNoSandboxFallback) { + auto pathDir = make_tempdir("mcpp-nasm-path-old"); + make_fake_nasm(pathDir, "NASM version 2.10.00 compiled on Jan 1 2020"); + + auto base = path_without_real_nasm(); + mcpp::platform::env::ScopedEnv path("PATH", pathDir.string() + ":" + base); + + auto home = make_tempdir("mcpp-nasm-home-empty"); // no sandbox nasm + mcpp::xlings::Env env{.home = home}; + + EXPECT_FALSE(mcpp::xlings::find_usable_nasm(env).has_value()); + + std::filesystem::remove_all(pathDir); + std::filesystem::remove_all(home); +} + +TEST(FindUsableNasm, FallsBackToSandboxWhenPathHasNoNasm) { + // PATH has the usual system directories (so the sandbox script's own + // shebang still resolves) but none of them contain a real `nasm`. + mcpp::platform::env::ScopedEnv path("PATH", path_without_real_nasm()); + + auto home = make_tempdir("mcpp-nasm-home-sandbox"); + auto sandboxDir = home / "data" / "xpkgs" / "xim-x-nasm" / "3.02"; + auto expected = make_fake_nasm(sandboxDir, "NASM version 3.02 compiled on Jan 1 2026"); + + mcpp::xlings::Env env{.home = home}; + + auto found = mcpp::xlings::find_usable_nasm(env); + ASSERT_TRUE(found.has_value()); + EXPECT_EQ(*found, expected); + + std::filesystem::remove_all(home); +} + +TEST(FindSandboxNasm, FindsUnderVersionedBinSubdir) { + auto home = make_tempdir("mcpp-nasm-home-bin-subdir"); + auto sandboxDir = home / "data" / "xpkgs" / "xim-x-nasm" / "3.02" / "bin"; + auto expected = make_fake_nasm(sandboxDir, "NASM version 3.02 compiled on Jan 1 2026"); + + mcpp::xlings::Env env{.home = home}; + + auto found = mcpp::xlings::find_sandbox_nasm(env); + ASSERT_TRUE(found.has_value()); + EXPECT_EQ(*found, expected); + + std::filesystem::remove_all(home); +} + +TEST(FindSandboxNasm, RejectsInstalledNasmBelowMinVersion) { + auto home = make_tempdir("mcpp-nasm-home-old-sandboxed"); + auto sandboxDir = home / "data" / "xpkgs" / "xim-x-nasm" / "2.10"; + make_fake_nasm(sandboxDir, "NASM version 2.10.00 compiled on Jan 1 2020"); + + mcpp::xlings::Env env{.home = home}; + + EXPECT_FALSE(mcpp::xlings::find_sandbox_nasm(env).has_value()); + + std::filesystem::remove_all(home); +} + +TEST(FindSandboxNasm, NoSandboxDirectoryYieldsNullopt) { + auto home = make_tempdir("mcpp-nasm-home-absent"); + mcpp::xlings::Env env{.home = home}; + + EXPECT_FALSE(mcpp::xlings::find_sandbox_nasm(env).has_value()); + + std::filesystem::remove_all(home); +} + +#endif // !defined(_WIN32) + TEST(XlingsIndexFreshness, RequiresDefaultMcpplibsIndex) { auto home = make_tempdir("mcpp-xlings-index-freshness"); std::filesystem::create_directories(home / "data" / "xim-pkgindex" / "pkgs"); From c044f8e05744a288d44018042c2146387f6d388d Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 20:05:32 +0800 Subject: [PATCH 13/21] feat(workspace): root-anchored path/index inheritance + default-namespace index redirect + `mcpp run -p ` (#224, R6) --- src/build/execute.cppm | 29 +++-- src/build/prepare.cppm | 46 ++++++-- src/cli.cppm | 2 + src/cli/cmd_build.cppm | 7 +- src/manifest/toml.cppm | 87 ++++++++++++++- src/pm/index_spec.cppm | 25 ++++- src/project.cppm | 39 +++++-- tests/e2e/120_ws_root_indices.sh | 138 ++++++++++++++++++++++++ tests/e2e/121_default_ns_redirect.sh | 146 +++++++++++++++++++++++++ tests/e2e/122_run_member.sh | 80 ++++++++++++++ tests/unit/test_manifest.cpp | 152 +++++++++++++++++++++++++++ 11 files changed, 726 insertions(+), 25 deletions(-) create mode 100755 tests/e2e/120_ws_root_indices.sh create mode 100755 tests/e2e/121_default_ns_redirect.sh create mode 100755 tests/e2e/122_run_member.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 0b65c8f4..6926e37d 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -584,21 +584,36 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, } // `mcpp run` driver: build, locate the binary target, exec it with the -// resolved runtime environment. +// resolved runtime environment. `package_filter` (`-p`/`--package`) scopes +// a workspace invocation to one member — single-member only, no +// `--workspace` fan-out (running N binaries in one invocation isn't a +// coherent "run"). Threaded straight to prepare_build's BuildOverrides, +// which already does the member switch (basename OR member path — the same +// rule mcpp::project::resolve_member_dir documents for build/test). export int build_run_target(const std::optional& targetName, - std::span passthrough) { + std::span passthrough, + const std::string& package_filter = {}) { // mcpp#225 (E2): reuse the resolved build cache when it's still fresh, // skipping prepare_build's toolchain resolution + modgraph scan - // entirely — mirrors cmd_build's try_fast_build fast path. - if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path())) { - if (auto rc = try_fast_run(*root, targetName, passthrough)) { - return *rc; + // entirely — mirrors cmd_build's try_fast_build fast path. The cached + // entry was written for whichever package occupied the project root + // last time; a `-p` filter always needs prepare_build's member switch, + // so skip the fast path in that case (mirrors cmd_build's fast-path + // bypass whenever ov.package_filter is set). + if (package_filter.empty()) { + if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path())) { + if (auto rc = try_fast_run(*root, targetName, passthrough)) { + return *rc; + } } } // Build first. Single prepare_build → drive build → reuse ctx to locate // the binary, so we don't re-resolve the toolchain or re-scan modgraph. - auto ctx = prepare_build(/*print_fp=*/false); + mcpp::build::BuildOverrides ov; + ov.package_filter = package_filter; + auto ctx = prepare_build(/*print_fp=*/false, /*includeDevDeps=*/false, + /*extraTargets=*/{}, ov); if (!ctx) { std::println(stderr, "error: {}", ctx.error()); return 2; } if (auto rc = run_build_plan(*ctx, /*verbose=*/false, /*no_cache=*/false); rc != 0) return rc; diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 44c73b24..cc09c8a1 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -577,8 +577,11 @@ prepare_build(bool print_fingerprint, if (!m) return std::unexpected(std::format( "workspace member '{}': {}", targetMember, m.error().format())); - // Merge workspace dependency versions - mcpp::project::merge_workspace_deps(*m, *wsManifest); + // Merge workspace dependency versions/paths. `*root` is still the + // WORKSPACE root here (the `root = memberDir` reassignment below + // hasn't happened yet), so it anchors any relative `path` in + // `[workspace.dependencies]` (#224). + mcpp::project::merge_workspace_deps(*m, *wsManifest, *root); // Inherit workspace toolchain if member doesn't define one if (m->toolchain.byPlatform.empty()) { @@ -590,9 +593,19 @@ prepare_build(bool print_fingerprint, m->targetOverrides[triple] = entry; } } - // Inherit workspace indices if member doesn't define any + // Inherit workspace indices if member doesn't define any. A + // relative `[indices].path` was declared at the WORKSPACE root, + // so it must resolve against `*root` (still the workspace root + // here), not the member directory — otherwise every member + // needs its own `../`-prefixed copy of the same declaration + // (#224). if (m->indices.empty() && !wsManifest->indices.empty()) { m->indices = wsManifest->indices; + for (auto& [_, idx] : m->indices) { + if (idx.is_local() && idx.path.is_relative()) { + idx.path = std::filesystem::weakly_canonical(*root / idx.path); + } + } } mcpp::ui::status("Workspace", std::format("building member '{}'", targetMember)); @@ -604,7 +617,9 @@ prepare_build(bool print_fingerprint, if (!wsRoot.empty()) { auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); if (wsm && wsm->workspace.present) { - mcpp::project::merge_workspace_deps(*m, *wsm); + // #224: anchor relative `path`/`[indices].path` to the + // workspace root, not this member's own directory. + mcpp::project::merge_workspace_deps(*m, *wsm, wsRoot); if (m->toolchain.byPlatform.empty()) { m->toolchain = wsm->toolchain; } @@ -616,6 +631,11 @@ prepare_build(bool print_fingerprint, // Inherit workspace indices if member doesn't define any if (m->indices.empty() && !wsm->indices.empty()) { m->indices = wsm->indices; + for (auto& [_, idx] : m->indices) { + if (idx.is_local() && idx.path.is_relative()) { + idx.path = std::filesystem::weakly_canonical(wsRoot / idx.path); + } + } } } } @@ -1164,7 +1184,14 @@ prepare_build(bool print_fingerprint, auto ns = spec.namespace_.empty() ? std::string(mcpp::pm::kDefaultNamespace) : spec.namespace_; - if (ns == mcpp::pm::kDefaultNamespace) return true; + if (ns == mcpp::pm::kDefaultNamespace) { + // R6: `[indices] default = {...}` (normalized to + // kDefaultNamespace by toml.cppm) redirects the default + // namespace away from the builtin registry — consult it + // before assuming builtin, same as any named namespace. + auto it = m->indices.find(std::string(mcpp::pm::kDefaultNamespace)); + return it == m->indices.end() || it->second.is_builtin(); + } auto it = m->indices.find(ns); if (it == m->indices.end()) return true; @@ -1324,7 +1351,14 @@ prepare_build(bool print_fingerprint, auto findIndexForNs = [&](const std::string& ns) -> const mcpp::pm::IndexSpec* { - if (ns.empty() || ns == std::string(mcpp::pm::kDefaultNamespace)) return nullptr; + if (ns.empty() || ns == std::string(mcpp::pm::kDefaultNamespace)) { + // R6: `[indices] default = {...}` (normalized to + // kDefaultNamespace by toml.cppm) redirects the default + // namespace — return it when present instead of unconditionally + // falling back to the builtin index. + auto it = m->indices.find(std::string(mcpp::pm::kDefaultNamespace)); + return it == m->indices.end() ? nullptr : &it->second; + } if (auto it = m->indices.find(ns); it != m->indices.end()) { return &it->second; } diff --git a/src/cli.cppm b/src/cli.cppm index 71e2db22..85f63df8 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -244,6 +244,8 @@ int run(int argc, char** argv) { // NB: this positional is a BINARY NAME from [[bin]]/src layout — // unrelated to `--target ` (the cross-target axis). .arg(cl::Arg("target").help("Binary name (optional)")) + .option(cl::Option("package").short_name('p').takes_value().value_name("NAME") + .help("Run only the named workspace member (single-member; no --workspace fan-out)")) .action(wrap_rc([&passthrough](const cl::ParsedArgs& p) { return cmd_run(p, std::span(passthrough)); }))) diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 198f3680..646e4ea3 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -103,7 +103,12 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, // is the optional binary target name. std::optional targetName; if (parsed.positional_count() > 0) targetName = parsed.positional(0); - return mcpp::build::build_run_target(targetName, passthrough); + // -p/--package : scope to one workspace member, same flag/rule + // as `mcpp build -p` / `mcpp test -p` (mcpp::project::resolve_member_dir). + // `mcpp run` is single-member only — no `--workspace` fan-out. + std::string package_filter; + if (auto p = parsed.value("package")) package_filter = *p; + return mcpp::build::build_run_target(targetName, passthrough, package_filter); } export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 0414dd27..6fe2aecd 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -953,16 +953,47 @@ std::expected parse_string(std::string_view content, continue; } if (!v.is_table()) continue; + auto& sub = v.as_table(); + // #224: a flat (non-namespaced, non-dotted) key whose value is + // an inline dep-spec table — `ylib = { path = "..." }` — must + // be recognized as ONE dependency spec here, same as + // load_deps_table's (1') branch does for [dependencies]. + // Without this check it falls into load_selector_dep_table + // below, which treats "path"/"version"/etc. as nested + // selector path components and silently mis-files the entry + // under a key like "ylib.path" instead of "ylib". + if (looks_like_inline_dep_spec(sub)) { + if (k.find('.') != std::string::npos) { + auto depKey = mcpp::pm::compat::split_legacy_dependency_key(k); + auto selector = mcpp::pm::make_direct_dependency_selector( + depKey.namespace_, depKey.shortName, k); + if (auto r = assign_dep("workspace.dependencies", + m.workspace.dependencies, + selector, v, + depKey.legacyDottedKey); !r) { + return std::unexpected(r.error()); + } + continue; + } + auto selector = mcpp::pm::resolve_dependency_selector( + k, mcpp::pm::DependencySelectorMode::OmittedMcpplibsPriority); + if (auto r = assign_dep("workspace.dependencies", + m.workspace.dependencies, + selector, v, false); !r) { + return std::unexpected(r.error()); + } + continue; + } if (is_namespace_table("workspace.dependencies", k)) { if (auto r = load_nested_dep_table("workspace.dependencies", m.workspace.dependencies, - k, k, v.as_table()); !r) { + k, k, sub); !r) { return std::unexpected(r.error()); } } else { if (auto r = load_selector_dep_table("workspace.dependencies", m.workspace.dependencies, - k, v.as_table()); !r) { + k, sub); !r) { return std::unexpected(r.error()); } } @@ -979,8 +1010,23 @@ std::expected parse_string(std::string_view content, // mcpplibs = { url = "https://...", rev = "abc123" } # pin built-in if (auto* indices_t = doc->get_table("indices")) { for (auto& [k, v] : *indices_t) { + // R6: `default` (canonical spelling) or the empty-quoted key `""` + // redirects the DEFAULT namespace (bare `gtest = "1.15.2"` deps, + // not routed through any explicit namespace prefix) rather than + // declaring a literal index named "default". Normalize both + // spellings to kDefaultNamespace so prepare.cppm's lookups + // (usesBuiltinIndex / findIndexForNs) key on the same string + // dependency resolution already uses for the default namespace. + bool isDefaultAlias = (k.empty() || k == "default"); + std::string key = isDefaultAlias ? std::string(kDefaultNamespace) : k; mcpp::pm::IndexSpec spec; - spec.name = k; + // `spec.name` mirrors the (normalized) map key: several call + // sites (config.cppm's ensure_project_index_dir, prepare.cppm's + // install-target formatting) key xlings' project-index identity + // off whichever of {map key, spec.name} they happen to iterate, + // so the two must always agree — same invariant as every other + // [indices] entry, "default"/"" included. + spec.name = key; if (v.is_string()) { // Short form: key = "url" @@ -1001,7 +1047,40 @@ std::expected parse_string(std::string_view content, "[indices].{} must be a string (url) or inline table", k))); } - m.indices[k] = std::move(spec); + // R6's design goal for the default-namespace redirect is + // pointing the default namespace at a LOCAL index checkout + // (`path = ...`), so the index repo itself can test module + // packages against an unpublished tree. A `url` form silently + // no-ops instead of redirecting: is_builtin() (index_spec.cppm) + // still returns true whenever `path` is empty, so every + // consumer (prepare.cppm's readLuaContent/useProjectEnv/ + // findRawInstalled, config.cppm's ensure_project_index_dir) + // falls through to the builtin/global registry and just + // ignores the configured url — no error, nothing redirected. + // Reject it loudly at parse time instead of accepting a no-op. + // This only fires for the `default`/`""` ALIAS; a literal + // `[indices] mcpplibs = { url = ..., rev = ... }` (pin the + // builtin registry to a commit) is a different key spelling + // and must keep working unchanged. + if (isDefaultAlias && spec.path.empty()) { + return std::unexpected(error(origin, std::format( + "[indices] default-namespace redirect currently supports only " + "'path = ...' (a local index checkout), not 'url'; got url = '{}'. " + "Use a named namespace for a remote custom index.", spec.url))); + } + + // `default`, `""`, and a literal `mcpplibs` key all normalize to + // the same map slot (kDefaultNamespace). Without a duplicate + // check, declaring more than one of these silently clobbers + // whichever was assigned last, order-dependent on TOML table + // iteration. Fail loudly instead. + if (m.indices.contains(key)) { + return std::unexpected(error(origin, std::format( + "[indices] '{}' collides with an existing default-namespace entry " + "(default/\"\"/mcpplibs all map to the same slot)", k))); + } + + m.indices[key] = std::move(spec); } } diff --git a/src/pm/index_spec.cppm b/src/pm/index_spec.cppm index a39c80eb..27110100 100644 --- a/src/pm/index_spec.cppm +++ b/src/pm/index_spec.cppm @@ -21,7 +21,30 @@ struct IndexSpec { bool is_local() const { return !path.empty(); } bool is_pinned() const { return !rev.empty(); } - bool is_builtin() const { return name == "mcpplibs"; } + // R6: `name == "mcpplibs"` alone used to mean "builtin" unconditionally, + // which was correct while the only way to reach that name was the + // literal `[indices] mcpplibs = { url = ..., rev = ... }` pin form (still + // routes through the shared global registry, just pinned to a commit — + // `path` is never set in that form). Since [indices] now also accepts + // `default = {...}` / `"" = {...}` as aliases for the SAME map entry + // (both normalize to the "mcpplibs" name — see toml.cppm's [indices] + // parse), a `path`-based redirect of the default namespace to a local + // checkout would otherwise be misidentified as "builtin" too, and get + // silently skipped by the project-index plumbing (ensure_project_index_dir + // / prepare.cppm's useProjectEnv) instead of actually redirecting. + // `path` set is an unambiguous signal that this entry does NOT point at + // the real upstream builtin registry, regardless of which spelling + // produced the "mcpplibs" name. + // + // Note: this only covers the `path` form. A `default`/`""`-alias entry + // with `url` (not `path`) would ALSO satisfy `path.empty()` here and be + // misidentified as builtin, silently no-opping the redirect — so + // toml.cppm's `[indices]` parser rejects that combination at parse time + // (loud error) instead of letting it reach this predicate. The literal + // `[indices] mcpplibs = { url = ..., rev = ... }` pin form is unaffected + // by that rejection and still resolves `is_builtin() == true` here, as + // intended. + bool is_builtin() const { return name == "mcpplibs" && path.empty(); } }; } // namespace mcpp::pm diff --git a/src/project.cppm b/src/project.cppm index cad11b6d..48ca2e70 100644 --- a/src/project.cppm +++ b/src/project.cppm @@ -48,24 +48,51 @@ export std::filesystem::path find_workspace_root(const std::filesystem::path& me return {}; } -// Merge workspace.dependencies versions into a member's deps. +// Merge workspace.dependencies into a member's deps (`x.workspace = true`). +// +// #224: this used to propagate only `version`, so a workspace-level +// `[workspace.dependencies] x = { path = "..." }` inherited by a member was +// silently treated as a version/index dep (empty version) and failed to +// resolve. Now the location fields (path/git/*) travel too — a dep spec is +// one of {version, path, git} so copying whichever the workspace declared +// is correct. +// +// `wsRoot` anchors a relative `path`: the workspace author wrote it +// relative to the WORKSPACE ROOT (where `[workspace.dependencies]` lives), +// not the inheriting member's own directory, so it is resolved to an +// absolute path here — downstream path-dep resolution (relative to the +// member root) then sees an already-absolute path and leaves it alone. export void merge_workspace_deps(mcpp::manifest::Manifest& member, - const mcpp::manifest::Manifest& workspace) { + const mcpp::manifest::Manifest& workspace, + const std::filesystem::path& wsRoot = {}) { + auto copy_from = [&](mcpp::manifest::DependencySpec& spec, + const mcpp::manifest::DependencySpec& wsSpec) { + spec.version = wsSpec.version; + spec.path = wsSpec.path; + spec.git = wsSpec.git; + spec.gitRev = wsSpec.gitRev; + spec.gitRefKind = wsSpec.gitRefKind; + if (!spec.path.empty() && !wsRoot.empty()) { + std::filesystem::path p(spec.path); + if (p.is_relative()) { + spec.path = std::filesystem::weakly_canonical(wsRoot / p).string(); + } + } + spec.inheritWorkspace = false; + }; auto merge_map = [&](std::map& deps) { for (auto& [name, spec] : deps) { if (!spec.inheritWorkspace) continue; // Try exact key match first auto it = workspace.workspace.dependencies.find(name); if (it != workspace.workspace.dependencies.end()) { - spec.version = it->second.version; - spec.inheritWorkspace = false; + copy_from(spec, it->second); continue; } // Try short name for default-ns deps auto shortIt = workspace.workspace.dependencies.find(spec.shortName); if (shortIt != workspace.workspace.dependencies.end()) { - spec.version = shortIt->second.version; - spec.inheritWorkspace = false; + copy_from(spec, shortIt->second); } } }; diff --git a/tests/e2e/120_ws_root_indices.sh b/tests/e2e/120_ws_root_indices.sh new file mode 100755 index 00000000..a5a2466b --- /dev/null +++ b/tests/e2e/120_ws_root_indices.sh @@ -0,0 +1,138 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# #224: root-anchored inheritance. +# 1. A workspace ROOT declares `[indices] x = { path = "..." }` (a +# relative path) ONCE. A member with NO [indices] section of its own +# inherits it and must resolve the relative path against the +# WORKSPACE ROOT, not `//...` — otherwise every member +# would need its own `../`-prefixed re-declaration. +# 2. `[workspace.dependencies] ylib = { path = "..." }` (also relative to +# the workspace root) is usable by a member via `ylib.workspace = true` +# — previously `merge_workspace_deps` only propagated `version`, so a +# path-form workspace dependency was silently dropped and failed to +# resolve. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +mkdir -p "$TMP/ws" +cd "$TMP/ws" + +# ── A project-local index, declared relative to the WORKSPACE ROOT ────── +mkdir -p local-index/pkgs/x +cat > local-index/pkgs/x/x.widget2.lua <<'EOF' +package = { + spec = "1", + namespace = "x", + name = "x.widget2", + description = "Namespaced package reachable only via the workspace-root-anchored index", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { + ["1.0.0"] = { + url = "https://example.invalid/widget2-1.0.0.tar.gz", + sha256 = "0000000000000000000000000000000000000000000000000000000000000000", + }, + }, + }, + mcpp = { + language = "c++23", + import_std = false, + sources = { "src/widget2.cppm" }, + targets = { ["widget2"] = { kind = "lib" } }, + deps = {}, + }, +} +EOF + +# ── A path-form workspace dependency, also relative to the WORKSPACE ROOT ── +mkdir -p ylib/src +cat > ylib/mcpp.toml <<'EOF' +[package] +name = "ylib" +version = "0.1.0" + +[targets.ylib] +kind = "lib" +EOF +cat > ylib/src/ylib.cppm <<'EOF' +export module ylib; + +export int ylib_value() { + return 7; +} +EOF + +# ── Workspace root: virtual, one member, no re-declaration required ───── +cat > mcpp.toml < member-a/.mcpp/.xlings/data/xpkgs/x.widget2/1.0.0/src/widget2.cppm <<'EOF' +export module widget2; + +export int widget2_value() { + return 35; +} +EOF + +# Deliberately NO [indices] section here — must inherit from the root and +# resolve "local-index" against the WORKSPACE ROOT, not member-a/local-index. +cat > member-a/mcpp.toml <<'EOF' +[package] +name = "member-a" +version = "0.1.0" + +[dependencies.x] +widget2 = "1.0.0" + +[dependencies] +ylib = { workspace = true } + +[targets.member-a] +kind = "bin" +main = "src/main.cpp" +EOF + +cat > member-a/src/main.cpp <<'EOF' +import widget2; +import ylib; + +int main() { + return (widget2_value() + ylib_value() == 42) ? 0 : 1; +} +EOF + +"$MCPP" build -p member-a > build.log 2>&1 || { + cat build.log + echo "FAIL: member did not resolve root-anchored [indices]/[workspace.dependencies] path" + exit 1 +} + +"$MCPP" run -p member-a > run.log 2>&1 || { + cat run.log + echo "FAIL: run failed" + exit 1 +} + +grep -q '\[package\."x.widget2"\]' member-a/mcpp.lock || { + cat member-a/mcpp.lock 2>/dev/null || true + echo "FAIL: expected x.widget2 lock entry" + exit 1 +} + +echo "OK" diff --git a/tests/e2e/121_default_ns_redirect.sh b/tests/e2e/121_default_ns_redirect.sh new file mode 100755 index 00000000..3b73e7bf --- /dev/null +++ b/tests/e2e/121_default_ns_redirect.sh @@ -0,0 +1,146 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# R6: `[indices] default = { path = ... }` (also spellable `"" = {...}`) +# redirects the DEFAULT namespace — bare `gizmo = "1.0.0"` deps with no +# namespace prefix — to a local checkout, instead of the two hardcoded +# short-circuits (prepare.cppm usesBuiltinIndex / findIndexForNs) that used +# to route the default namespace straight to the builtin index regardless +# of [indices]. The package is declared ONLY in a project-local index dir +# that lives outside $MCPP_HOME (never seeded into the global/builtin +# registry location), so a build that still fell through to the builtin +# short-circuit would fail to resolve it. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +# A local index directory, deliberately NOT under $MCPP_HOME — it must be +# reached only via the [indices] default = {...} redirect. +INDEX_DIR="$TMP/local-index" +mkdir -p "$INDEX_DIR/pkgs/g" +cat > "$INDEX_DIR/pkgs/g/gizmo.lua" <<'EOF' +package = { + spec = "1", + name = "gizmo", + description = "Default-namespace package served only via a redirected index", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { + ["1.0.0"] = { + url = "https://example.invalid/gizmo-1.0.0.tar.gz", + sha256 = "0000000000000000000000000000000000000000000000000000000000000000", + }, + }, + }, + mcpp = { + language = "c++23", + import_std = false, + sources = { "src/gizmo.cppm" }, + targets = { ["gizmo"] = { kind = "lib" } }, + deps = {}, + }, +} +EOF + +mkdir -p "$TMP/project/app/src" \ + "$TMP/project/app/.mcpp/.xlings/data/xpkgs/gizmo/1.0.0/src" +cd "$TMP/project/app" + +cat > .mcpp/.xlings/data/xpkgs/gizmo/1.0.0/src/gizmo.cppm <<'EOF' +export module gizmo; + +export int gizmo_value() { + return 42; +} +EOF + +cat > src/main.cpp <<'EOF' +import gizmo; + +int main() { + return gizmo_value() == 42 ? 0 : 1; +} +EOF + +cat > mcpp.toml < build.log 2>&1 || { + cat build.log + echo "FAIL: build did not resolve default-namespace dep from the redirected local index" + exit 1 +} + +"$MCPP" run > run.log 2>&1 || { + cat run.log + echo "FAIL: run failed" + exit 1 +} + +grep -q '\[package\."gizmo"\]' mcpp.lock || { + cat mcpp.lock + echo "FAIL: expected gizmo package lock entry" + exit 1 +} + +# ── "" as an alternate spelling of the redirect key ───────────────────── +mkdir -p "$TMP/project/app2/src" +cd "$TMP/project/app2" + +cat > src/main.cpp <<'EOF' +import gizmo; + +int main() { + return gizmo_value() == 42 ? 0 : 1; +} +EOF + +cat > mcpp.toml < .mcpp/.xlings/data/xpkgs/gizmo/1.0.0/src/gizmo.cppm <<'EOF' +export module gizmo; + +export int gizmo_value() { + return 42; +} +EOF + +"$MCPP" build > build.log 2>&1 || { + cat build.log + echo 'FAIL: [indices] "" = {...} did not redirect the default namespace' + exit 1 +} + +echo "OK" diff --git a/tests/e2e/122_run_member.sh b/tests/e2e/122_run_member.sh new file mode 100755 index 00000000..1e1e93a6 --- /dev/null +++ b/tests/e2e/122_run_member.sh @@ -0,0 +1,80 @@ +#!/usr/bin/env bash +# requires: +# run-member-selection: `mcpp run` gains -p/--package, matching `build -p` / +# `test -p`. `mcpp run -p memberB` in a workspace with two binary members +# must build+run memberB's binary specifically (not memberA's, not an +# auto-picked one). +set -euo pipefail + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p memberA/src memberB/src + +cat > mcpp.toml <<'EOF' +[workspace] +members = ["memberA", "memberB"] +EOF + +cat > memberA/mcpp.toml <<'EOF' +[package] +name = "memberA" +version = "0.1.0" + +[targets.memberA] +kind = "bin" +main = "src/main.cpp" +EOF +cat > memberA/src/main.cpp <<'EOF' +import std; +int main() { std::println("hello from memberA"); return 0; } +EOF + +cat > memberB/mcpp.toml <<'EOF' +[package] +name = "memberB" +version = "0.1.0" + +[targets.memberB] +kind = "bin" +main = "src/main.cpp" +EOF +cat > memberB/src/main.cpp <<'EOF' +import std; +int main() { std::println("hello from memberB"); return 0; } +EOF + +# `-p memberB` must run memberB's binary — assert its DISTINCT output, not +# memberA's (and not "no binary target found"). +OUT=$("$MCPP" run -p memberB 2>run_b.log) || { cat run_b.log; echo "FAIL: run -p memberB failed"; exit 1; } +echo "$OUT" +[[ "$OUT" == *"hello from memberB"* ]] || { + echo "FAIL: expected memberB's output, got: $OUT" + exit 1 +} +[[ "$OUT" != *"hello from memberA"* ]] || { + echo "FAIL: memberA's output leaked into -p memberB run" + exit 1 +} + +# `-p memberA` runs the other one. +OUT=$("$MCPP" run -p memberA 2>run_a.log) || { cat run_a.log; echo "FAIL: run -p memberA failed"; exit 1; } +echo "$OUT" +[[ "$OUT" == *"hello from memberA"* ]] || { + echo "FAIL: expected memberA's output, got: $OUT" + exit 1 +} + +# Unknown member name errors clearly instead of silently picking one. +if "$MCPP" run -p nope > run_bad.log 2>&1; then + echo "FAIL: run -p nope should have failed" + exit 1 +fi +grep -qi "nope" run_bad.log || { + cat run_bad.log + echo "FAIL: expected an error mentioning the unknown member 'nope'" + exit 1 +} + +echo "OK" diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 270e1194..3d4d93d5 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -1041,6 +1041,158 @@ mbedtls = { workspace = true } EXPECT_EQ(s.shortName, "mbedtls"); } +TEST(Manifest, IndicesDefaultKeyNormalizesToDefaultNamespace) { + // R6: `[indices] default = {...}` (the canonical spelling) is a + // redirect for the DEFAULT namespace (bare `gtest = "1.15.2"` deps), + // not a literal index named "default". It must be stored under + // kDefaultNamespace so the two short-circuits in prepare.cppm + // (usesBuiltinIndex / findIndexForNs) can find it. + constexpr auto src = R"( +[package] +name = "hello" +version = "0.1.0" +[language] +standard = "c++23" +[modules] +sources = ["src/**/*.cppm"] +[indices] +default = { path = "../local-pkgs" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + auto it = m->indices.find(std::string(mcpp::manifest::kDefaultNamespace)); + ASSERT_NE(it, m->indices.end()) + << "[indices] default = {...} must normalize to kDefaultNamespace"; + EXPECT_EQ(it->second.path.string(), "../local-pkgs"); + EXPECT_FALSE(it->second.is_builtin()); + EXPECT_EQ(m->indices.find("default"), m->indices.end()) + << "the literal key \"default\" must not survive normalization"; +} + +TEST(Manifest, IndicesEmptyStringKeyNormalizesToDefaultNamespace) { + // The empty-quoted key `""` is also accepted as a spelling of the + // default-namespace redirect. + constexpr auto src = R"( +[package] +name = "hello" +version = "0.1.0" +[language] +standard = "c++23" +[modules] +sources = ["src/**/*.cppm"] +[indices] +"" = { path = "../local-pkgs" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + auto it = m->indices.find(std::string(mcpp::manifest::kDefaultNamespace)); + ASSERT_NE(it, m->indices.end()) + << "[indices] \"\" = {...} must normalize to kDefaultNamespace"; + EXPECT_EQ(it->second.path.string(), "../local-pkgs"); +} + +TEST(Manifest, IndicesDefaultKeyWithUrlIsRejected) { + // R6's design goal is redirecting the default namespace to a LOCAL + // index checkout (`path = ...`), so the index repo can test module + // packages. The `url` form is not a design requirement, and silently + // accepting it is worse than rejecting it: is_builtin() (index_spec + // .cppm) treats any "mcpplibs"-named entry with an empty `path` as + // builtin, so a `default = { url = ... }` redirect would silently + // no-op — every consumer falls through to the real builtin registry + // and the configured url is just ignored, with no error surfaced. + // Reject it loudly at parse time instead. + constexpr auto src = R"( +[package] +name = "hello" +version = "0.1.0" +[language] +standard = "c++23" +[modules] +sources = ["src/**/*.cppm"] +[indices] +default = { url = "https://example.com/mcpp-index.git" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_FALSE(m.has_value()) + << "[indices] default = { url = ... } must be rejected, not silently " + "accepted as a no-op redirect"; + EXPECT_NE(m.error().format().find("path"), std::string::npos) + << "error should mention that only 'path' is supported for the " + "default-namespace redirect; got: " << m.error().format(); +} + +TEST(Manifest, IndicesMcpplibsUrlPinStillAccepted) { + // A literal `[indices] mcpplibs = { url = ..., rev = ... }` is a + // DIFFERENT key spelling from the `default`/`""` alias — it pins the + // real builtin registry to a commit, not a default-namespace redirect + // — and must keep working even though it also normalizes to the + // kDefaultNamespace map slot and also carries a `url`. + constexpr auto src = R"( +[package] +name = "hello" +version = "0.1.0" +[language] +standard = "c++23" +[modules] +sources = ["src/**/*.cppm"] +[indices] +mcpplibs = { url = "https://example.com/mcpp-index.git", rev = "abc123" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + auto it = m->indices.find(std::string(mcpp::manifest::kDefaultNamespace)); + ASSERT_NE(it, m->indices.end()); + EXPECT_EQ(it->second.url, "https://example.com/mcpp-index.git"); + EXPECT_EQ(it->second.rev, "abc123"); + EXPECT_TRUE(it->second.is_builtin()); +} + +TEST(Manifest, IndicesDefaultAndMcpplibsCollisionRejected) { + // `default`, `""`, and a literal `mcpplibs` key all normalize to the + // same map slot (kDefaultNamespace). Declaring more than one must be a + // parse error, not a silent last-writer-wins clobber. + constexpr auto src = R"( +[package] +name = "hello" +version = "0.1.0" +[language] +standard = "c++23" +[modules] +sources = ["src/**/*.cppm"] +[indices] +mcpplibs = { path = "../local-pkgs-a" } +default = { path = "../local-pkgs-b" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_FALSE(m.has_value()) + << "[indices] mcpplibs + default together must be rejected as a " + "collision on the same default-namespace slot"; + EXPECT_NE(m.error().format().find("collide"), std::string::npos) + << "error should mention the collision; got: " << m.error().format(); +} + +TEST(Manifest, WorkspaceDependenciesPathFormFlatKey) { + // #224: `[workspace.dependencies] ylib = { path = "ylib" }` — a plain + // (non-namespaced, non-dotted) top-level key with an inline path table + // — must parse to a path-form DependencySpec, not get misrouted into + // the namespace/selector-table parsers (which would treat "path" as a + // nested selector key). + constexpr auto src = R"( +[workspace] +members = ["a"] + +[workspace.dependencies] +ylib = { path = "ylib" } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + ASSERT_TRUE(m->workspace.dependencies.contains("ylib")) + << "workspace.dependencies must contain a flat 'ylib' key"; + auto& s = m->workspace.dependencies.at("ylib"); + EXPECT_EQ(s.path, "ylib"); + EXPECT_TRUE(s.version.empty()); +} + TEST(Manifest, NoWorkspaceSectionMeansNotPresent) { constexpr auto src = R"( [package] From 69e38eee94f8cbb4a916380bd6bfbfa436b856ef Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 20:22:34 +0800 Subject: [PATCH 14/21] =?UTF-8?q?docs(plan):=20C6+C7=20(=E7=B0=87F/?= =?UTF-8?q?=E7=B0=87G)=20done=20on=20=CE=B2=20=E2=80=94=20=CE=B2=20track?= =?UTF-8?q?=20complete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...0.0.97-architectural-remediation-implementation-plan.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index 1b043555..8b8e635b 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -21,9 +21,10 @@ | C3 簇A #226 #234 | ✅ 完成 `b71d93c` | token-list 两 pass:normalize_include_flags(全 include 族 joined/separated)+shell_quote_arg;修 1 Critical=MSVC include_dirs 需先绝对化再加 dialect 前缀(typed path 通道≠flag-string 通道);+5 单测+e2e 116;review 通过 | | C4 簇C #233 #235 | ✅ 完成 `3b4f18e`(α) | 对象路径:保留 basename 碰撞门,消歧前缀改 relPath.parent_path()(非碰撞项字节不变,4 个断言扁平路径 e2e 保绿)+唯一性断言;depfile:cxx_module/object 加 -MMD,过滤 GCC -fmodules 反向规则(awk 保留文本 include 记录,POSIX 门控);+2 e2e 117/118+1 单测;review 通过 | | C5 簇D #229 | ✅ 完成 `d6b3b6c`(β) | 3-site 漏斗(root+version-dep 保留,新增 path/git-dep site=真正 #229 修复);偏离 brief 有据(apply()-loop 会丢 cfg flags:makePackageRoot 早快照 privateBuild);+e2e 119;review 通过(3 风险全清) | -| C6 簇F #232 | ⏳ 进行中(β) | | -| C7 簇G #224 R6 +run-p | ☐ 待办 | | -| C8 版本 0.0.97 | ☐ 待办 | | +| C6 簇F #232 | ✅ 完成 `482b428`(β) | nasm 走工具链同款 resolve_xpkg_path("xim:nasm@3.02",autoInstall) 同步门 + guard 不再吞 config 错;ensure_nasm 退役→find_usable_nasm/find_sandbox_nasm;+6 单测+e2e 105 冷环境;review 通过 | +| C7 簇G #224 R6 +run-p | ✅ 完成 `c044f8e`(β) | #224 根锚点继承(merge_workspace_deps 传 path+按 wsRoot 解析;[indices] 相对路径按根)+ R6 默认命名空间 `default={path}` 重定向(两处短路查表;url-form 拒绝 loud+dup-key 拒绝)+ `run -p member`;+7 单测+e2e 120/121/122;review 通过(1 Important 已修) | +| C8 版本 0.0.97 | ⏳ 合并后进行 | | +| 合并 β→α | ⏳ 进行中 | merge-tree 预览无冲突 | | PR / CI / 合入 / release | ☐ 待办 | | ## 全局约束(Global Constraints) From 964f9d0b0724d3749c5e677b05e37498242e127a Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 20:29:15 +0800 Subject: [PATCH 15/21] chore(release): bump mcpp 0.0.96 -> 0.0.97 (scanner scope, flag model, build-graph identity, cfg funnel, nasm gate, workspace/R6) Closes #224 #225 #226 #227 #228 #229 #232 #233 #234 #235 + R6. #230 already fixed in 0.0.96; #215 deferred (upstream P2996). --- CHANGELOG.md | 25 +++++++++++++++++++++++++ mcpp.toml | 2 +- src/toolchain/fingerprint.cppm | 2 +- 3 files changed, 27 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 71f69ab3..928ce3c5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,31 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [0.0.97] — 2026-07-18 + +> 架构级修复批次(单 PR,逐簇 commit):ffmpeg-m/opencv-m 全源码直编暴露的第二层缺口 + workspace 测试基建语法空洞。 + +### 新增 + +- **`[[build.flags]]` 数组表写法**(#227):`[build].flags` 现同时接受 TOML 标准数组表 `[[build.flags]]` 与内联表数组两种等价写法(声明顺序=应用顺序不变),长 per-glob flags 条目不再挤单行。纯解析层扩展(自研 TOML parser 补全 array-of-tables);并加清单层 closed-grammar 守卫——非白名单段落误写 `[[x]]`(如 `[[dependencies]]` 手滑)现**硬报错**而非静默丢数据。 +- **glob 花括号交替 `{a,b}`**(#228):sources/flags glob 支持 `libavcodec/{aac,bsf,hevc}/**` 笛卡尔展开(嵌套/多组均可),vendored 大库 per-glob 声明不再重复。 +- **默认命名空间索引重定向**(R6):`[indices] default = { path = "..." }`(亦接受空引号键 `""`)可把默认命名空间(`namespace = ""` 的模块包)指向本地 checkout,补上此前只能重定向具名命名空间的语法空洞——使 index 仓能以 `mcpp test --workspace` 声明式验证 imgui/ffmpeg/opencv 等模块包,替代逐包 smoke shell。(`url` 形式的默认命名空间重定向暂不支持,解析期显式报错而非静默失效。) +- **`mcpp run -p `**:run 命令支持选择 workspace 成员并运行其二进制(与 `build`/`test` 的 `-p` 对齐)。 + +### 修复 + +- **源发现全树遍历致 `mcpp run` 前慢 + 缓存复用**(#225):glob 遍历改从字面前缀起(`src/**` 从 `src/` 起走,不再从项目根扫全树),并排除 `.git`/`target`/git 子模块边界;`mcpp run` 复用 `mcpp build` 已解析的缓存,不再每次重扫大子模块(此前含大 `compat/` 子模块时每次 ~9.5s)。 +- **相对 include 族 flag 未按项目根重写 + 含空格值未转义**(#226 #234):`-iquote`/`-isystem`/`-idirafter`/`-iprefix`/`-L` 现与 `-I` 一样按项目根绝对化(joined 与 separated 两拼写);`defines = ["T=long long"]` 等含空格值发射时 shell 引号保护,不再被拆成孤立参数。含 `[build] include_dirs` 在 MSVC 方言下的相对路径绝对化亦一并修正。 +- **对象路径按父目录名折叠致同名源冲突**(#233):不同目录同名源(`a/src/util.cpp` vs `b/src/util.cpp`,OpenCV/LLVM 式 `modules//src/*` 布局)对象路径改按碰撞时镜像源**相对路径**消歧 + 构建后唯一性断言;非碰撞项路径字节不变。 +- **模块 purview 内文本 `#include` 改动不触发重编**(#235):编译边补 depfile 追踪(过滤 GCC `-fmodules` 注入的反向规则以规避 ninja `inputs may not also have inputs`),purview/GMF/普通头改动均正确触发重编——顺带根治此前非 MSVC 下普通头改动也不重编的潜伏问题。 +- **依赖包 cfg 条件 sources 被消费时不展开**(#229):path/git 依赖的 `[target.'cfg(...)'.build].sources` 现与 root/version-dep 同样按已解析 target 求值(`mcpp build` 与 `mcpp test` 双路径),不再 undefined reference(与 #218 同类,收敛为统一 per-package 求值)。 +- **nasm 冷环境惰性自举时序**(#232):nasm 供给改走工具链同款同步门 `Fetcher::resolve_xpkg_path("xim:nasm@3.02", autoInstall=true)`(索引刷新前置 + 硬报错 + payload 校验),不再先判死后台补装;config 自举错误不再被 `if(cfg)` 门吞成误导性的 "no usable nasm"。 +- **workspace 根配置无法一次声明全员可用**(#224):`[workspace.dependencies]` 的 path 依赖可被成员经 `.workspace = true` 继承;根 `[indices]` 的相对 `path` 按 **workspace 根**解析(而非消费成员目录),成员不再需重复声明各自 `../` 相对路径。 + +### 备注 + +- #230(windows workspace 崩溃)已于 0.0.96 修复。#215(cppfly Clang 反射行)待上游 Clang 落地 P2996,不排期。 + ## [0.0.96] — 2026-07-18 ### 修复 diff --git a/mcpp.toml b/mcpp.toml index 777d868d..06a1bef4 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "0.0.96" +version = "0.0.97" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 015e69d8..27a324bc 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -18,7 +18,7 @@ import mcpp.toolchain.detect; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "0.0.96"; +inline constexpr std::string_view MCPP_VERSION = "0.0.97"; struct FingerprintInputs { Toolchain toolchain; From f0c19fb05a4627a846505c6fad5a4580bf38560f Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 20:31:53 +0800 Subject: [PATCH 16/21] =?UTF-8?q?docs(plan):=20=CE=B2=E2=86=92=CE=B1=20mer?= =?UTF-8?q?ged,=20C8=20released=20(0.0.97),=20verification=20green?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ....0.97-architectural-remediation-implementation-plan.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index 8b8e635b..781ee740 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -23,9 +23,11 @@ | C5 簇D #229 | ✅ 完成 `d6b3b6c`(β) | 3-site 漏斗(root+version-dep 保留,新增 path/git-dep site=真正 #229 修复);偏离 brief 有据(apply()-loop 会丢 cfg flags:makePackageRoot 早快照 privateBuild);+e2e 119;review 通过(3 风险全清) | | C6 簇F #232 | ✅ 完成 `482b428`(β) | nasm 走工具链同款 resolve_xpkg_path("xim:nasm@3.02",autoInstall) 同步门 + guard 不再吞 config 错;ensure_nasm 退役→find_usable_nasm/find_sandbox_nasm;+6 单测+e2e 105 冷环境;review 通过 | | C7 簇G #224 R6 +run-p | ✅ 完成 `c044f8e`(β) | #224 根锚点继承(merge_workspace_deps 传 path+按 wsRoot 解析;[indices] 相对路径按根)+ R6 默认命名空间 `default={path}` 重定向(两处短路查表;url-form 拒绝 loud+dup-key 拒绝)+ `run -p member`;+7 单测+e2e 120/121/122;review 通过(1 Important 已修) | -| C8 版本 0.0.97 | ⏳ 合并后进行 | | -| 合并 β→α | ⏳ 进行中 | merge-tree 预览无冲突 | -| PR / CI / 合入 / release | ☐ 待办 | | +| 合并 β→α | ✅ 完成 `c7a2b61` | merge --no-ff,无冲突;合并后全量 rebuild+test 35/35 绿;9 个新 e2e(114-122)全绿 | +| C8 版本 0.0.97 | ✅ 完成 `964f9d0` | fingerprint.cppm + mcpp.toml → 0.0.97;CHANGELOG 追加;`mcpp --version`=0.0.97 | +| 全量 e2e 回归 | ⏳ 后台运行 | run_all.sh @ 0.0.97 | +| 最终整支 review | ⏳ 进行中 | opus,cross-cutting | +| PR / CI / 合入 / release | ☐ 待办 | gh 已认证(Sunrisepeak),origin=mcpp-community/mcpp | ## 全局约束(Global Constraints) From d3ad22388e1355178ebb88ad620a939c0b04cb31 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 20:54:48 +0800 Subject: [PATCH 17/21] =?UTF-8?q?fix(flags):=20do=20not=20re-quote=20mcpp-?= =?UTF-8?q?generated=20link=20flags=20=E2=80=94=20restore=20rpath=20$ORIGI?= =?UTF-8?q?N?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit C3's shell_quote_arg (#234, for compile-time defines-with-spaces) was applied to per-unit LINK flags too, double-quoting the already-escaped rpath token `-Wl,-rpath,'$$ORIGIN'` → literal `'$ORIGIN'` in RUNPATH → dependency .so's next to the exe couldn't be resolved (e2e 55/56/57/64 regressed vs 0.0.96). join_flags gains shellQuote (default true for user compile flags); the link-flag join passes false. Regression test locks it. --- src/build/ninja_backend.cppm | 15 ++++++++++++--- tests/unit/test_ninja_backend.cpp | 32 +++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 3 deletions(-) diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index f114e27e..fb45e0e6 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -96,7 +96,8 @@ std::string local_include_flags(const CompileUnit& cu) { return flags; } -std::string join_flags(const std::vector& flags) { +std::string join_flags(const std::vector& flags, + bool shellQuote = true) { // mcpp#234: each vector element is already one argv token (e.g. a // manifest define `T=long long` arrives here as the single element // `-DT=long long`, pushed whole by apply_glob_flags) — but joining with @@ -104,10 +105,18 @@ std::string join_flags(const std::vector& flags) { // words once ninja handed the resolved command line to the shell. // shell_quote_arg is a no-op for tokens with nothing shell-significant // (`-std=c++23`, `-O2`, ...), so plain flags are untouched. + // + // shellQuote=false for mcpp-GENERATED link flags (lu.linkFlags): those + // are already correctly shell-quoted + ninja-escaped at construction + // (e.g. the rpath token `-Wl,-rpath,'$$ORIGIN'` from plan.cppm — single + // quotes protect it from shell $-expansion, `$$` is ninja's literal `$`). + // Re-running shell_quote_arg over such a token DOUBLE-quotes it, baking a + // literal `'$ORIGIN'` (quotes included) into the binary's RUNPATH so the + // dynamic linker can't resolve dependency .so's placed next to the exe. std::string out; for (auto const& flag : flags) { out += ' '; - out += shell_quote_arg(flag); + out += shellQuote ? shell_quote_arg(flag) : flag; } return out; } @@ -889,7 +898,7 @@ std::string emit_ninja_string(const BuildPlan& plan) { // binaries run on the build host and use the system -lc++, // distributable targets get the static LLVM libc++. See // CompileFlags::ldStdlibDefault/ldStdlibTest. - std::string unit = join_flags(lu.linkFlags); + std::string unit = join_flags(lu.linkFlags, /*shellQuote=*/false); unit += (lu.kind == mcpp::build::LinkUnit::TestBinary) ? flags.ldStdlibTest : flags.ldStdlibDefault; if (!unit.empty()) diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index 2267be2b..743af2c5 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -324,6 +324,38 @@ TEST(NinjaBackend, PlainFlagsPassThroughUnquoted) { << ninja; } +// Regression: mcpp-GENERATED per-unit LINK flags are already correctly +// shell-quoted + ninja-escaped at construction — e.g. the shared-dep rpath +// token `-Wl,-rpath,'$$ORIGIN'` (single quotes stop shell $-expansion, `$$` +// is ninja's literal `$`). join_flags for link flags must NOT re-run +// shell_quote_arg over them: doing so double-quotes the token, baking a +// literal `'$ORIGIN'` (quotes included) into the binary's RUNPATH so the +// dynamic linker can't find dependency .so's next to the exe (e2e 55-57/64). +TEST(NinjaBackend, LinkFlagsAreNotReQuoted) { + auto plan = minimal_plan(); + plan.compileUnits.push_back({ + .source = "src/main.cpp", + .object = "obj/main.o", + .packageName = "rpath_test", + }); + plan.linkUnits.push_back({ + .targetName = "app", + .kind = mcpp::build::LinkUnit::Binary, + .objects = {"obj/main.o"}, + .linkFlags = {"-Wl,-rpath,'$$ORIGIN'"}, + .output = "bin/app", + .entryMain = "src/main.cpp", + }); + + auto ninja = emit_ninja_string(plan); + + // The rpath token passes through verbatim (ninja `$$` = literal `$`). + EXPECT_NE(ninja.find("-Wl,-rpath,'$$ORIGIN'"), std::string::npos) << ninja; + // Must NOT be double-quoted: shell_quote_arg escaping an embedded `'` + // produces the `'\''` sequence, which only appears if it re-quoted. + EXPECT_EQ(ninja.find("'\\''"), std::string::npos) << ninja; +} + TEST(NinjaBackend, RootPackageCxxflagsAreEmittedOncePerUnit) { auto plan = minimal_plan(); plan.manifest.buildConfig.cxxflags = {"-DROOT_FLAG=1"}; From 179f83c7e97b6955db4c624605f3becaca787e23 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 20:55:50 +0800 Subject: [PATCH 18/21] docs(plan): full e2e passed (rpath regression fixed), final review READY --- ...0.0.97-architectural-remediation-implementation-plan.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md index 781ee740..f5428f33 100644 --- a/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md +++ b/.agents/docs/2026-07-18-v0.0.97-architectural-remediation-implementation-plan.md @@ -25,9 +25,10 @@ | C7 簇G #224 R6 +run-p | ✅ 完成 `c044f8e`(β) | #224 根锚点继承(merge_workspace_deps 传 path+按 wsRoot 解析;[indices] 相对路径按根)+ R6 默认命名空间 `default={path}` 重定向(两处短路查表;url-form 拒绝 loud+dup-key 拒绝)+ `run -p member`;+7 单测+e2e 120/121/122;review 通过(1 Important 已修) | | 合并 β→α | ✅ 完成 `c7a2b61` | merge --no-ff,无冲突;合并后全量 rebuild+test 35/35 绿;9 个新 e2e(114-122)全绿 | | C8 版本 0.0.97 | ✅ 完成 `964f9d0` | fingerprint.cppm + mcpp.toml → 0.0.97;CHANGELOG 追加;`mcpp --version`=0.0.97 | -| 全量 e2e 回归 | ⏳ 后台运行 | run_all.sh @ 0.0.97 | -| 最终整支 review | ⏳ 进行中 | opus,cross-cutting | -| PR / CI / 合入 / release | ☐ 待办 | gh 已认证(Sunrisepeak),origin=mcpp-community/mcpp | +| 全量 e2e 回归 | ✅ 通过(修 1 回归) | 119→123 pass;抓到并修 rpath `$ORIGIN` 双重引号回归(`d3ad223`,C3 shell_quote_arg 误碰 link flags);余 22/54/62 为沙箱环境性(gcc ELF interp 缺失,0.0.96 基线同败,CI 真工具链过) | +| 最终整支 review | ✅ 通过 `adc7c...` | opus,READY,无 Critical/Important;deferred minors 已分诊 | +| rpath 回归修复 | ✅ 完成 `d3ad223` | join_flags 加 shellQuote 参数,link flags 不再二次引号;+单测 LinkFlagsAreNotReQuoted;e2e 55/56/57/64 红→绿 | +| PR / CI / 合入 / release | ⏳ 进行中 | gh 已认证(Sunrisepeak),origin=mcpp-community/mcpp | ## 全局约束(Global Constraints) From 3340e3cb83a1e656cbd137059e9e090a660e82de Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 21:07:51 +0800 Subject: [PATCH 19/21] fix(flags): shell-quote only define-values, pass raw/link flags verbatim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Generalizes d3ad223: C3's shell_quote_arg (#234) was applied to ALL per-unit flags, breaking two kinds of element that legitimately contain a token-boundary space — raw descriptor flags (compat.lua `-include mcpp_lua_platform_config.h` → gcc saw one malformed arg → aarch64/macos/windows cross-build FAILED) and mcpp-generated pre-escaped link flags (rpath `$ORIGIN`). join_flags now quotes ONLY `-D`/`/D` define tokens whose value holds a space (the sole producer of an intra-value space); everything else is verbatim = 0.0.96 tokenization. +regression test RawMultiTokenFlagIsNotQuoted; LinkFlagsAreNotReQuoted + QuotesFlagValueWithSpace still green. --- src/build/ninja_backend.cppm | 43 ++++++++++++++++++------------- tests/unit/test_ninja_backend.cpp | 26 +++++++++++++++++++ 2 files changed, 51 insertions(+), 18 deletions(-) diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index fb45e0e6..73ab5419 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -96,27 +96,34 @@ std::string local_include_flags(const CompileUnit& cu) { return flags; } -std::string join_flags(const std::vector& flags, - bool shellQuote = true) { - // mcpp#234: each vector element is already one argv token (e.g. a - // manifest define `T=long long` arrives here as the single element - // `-DT=long long`, pushed whole by apply_glob_flags) — but joining with - // a bare space and no quoting let the embedded space split it into two - // words once ninja handed the resolved command line to the shell. - // shell_quote_arg is a no-op for tokens with nothing shell-significant - // (`-std=c++23`, `-O2`, ...), so plain flags are untouched. +std::string join_flags(const std::vector& flags) { + // mcpp#234: a manifest `defines = ["T=long long"]` arrives as the single + // element `-DT=long long` (pushed whole by apply_glob_flags) — a space + // that is genuinely PART of one argv token. Joining with a bare space let + // the shell split it into two words (`-DT=long` + `long`) once ninja + // handed the command line to the shell, so a define value with a space + // must be shell-quoted. // - // shellQuote=false for mcpp-GENERATED link flags (lu.linkFlags): those - // are already correctly shell-quoted + ninja-escaped at construction - // (e.g. the rpath token `-Wl,-rpath,'$$ORIGIN'` from plan.cppm — single - // quotes protect it from shell $-expansion, `$$` is ninja's literal `$`). - // Re-running shell_quote_arg over such a token DOUBLE-quotes it, baking a - // literal `'$ORIGIN'` (quotes included) into the binary's RUNPATH so the - // dynamic linker can't resolve dependency .so's placed next to the exe. + // But quote ONLY that case. Every OTHER flag element is passed through + // verbatim, because two other kinds of element legitimately contain a + // space that MUST stay a token boundary, and quoting them breaks the build: + // • raw descriptor flags pack multiple argv tokens into one string + // (compat.lua's `-include mcpp_lua_platform_config.h` — the space + // separates `-include` from its argument; quoting made gcc see one + // malformed arg → "No such file"; broke aarch64/macos/windows builds); + // • mcpp-generated link flags are already shell-quoted + ninja-escaped + // (`-Wl,-rpath,'$$ORIGIN'` — single quotes stop shell $-expansion, + // `$$` is ninja's literal `$`); re-quoting baked a literal `'$ORIGIN'` + // into RUNPATH so dependency .so's next to the exe couldn't resolve. + // The `defines` channel is the ONLY producer of a `-D`/`/D`-prefixed token + // whose space is intra-value, so gate the quoting on exactly that shape. std::string out; for (auto const& flag : flags) { out += ' '; - out += shellQuote ? shell_quote_arg(flag) : flag; + const bool defineWithSpace = + (flag.starts_with("-D") || flag.starts_with("/D")) + && flag.find(' ') != std::string::npos; + out += defineWithSpace ? shell_quote_arg(flag) : flag; } return out; } @@ -898,7 +905,7 @@ std::string emit_ninja_string(const BuildPlan& plan) { // binaries run on the build host and use the system -lc++, // distributable targets get the static LLVM libc++. See // CompileFlags::ldStdlibDefault/ldStdlibTest. - std::string unit = join_flags(lu.linkFlags, /*shellQuote=*/false); + std::string unit = join_flags(lu.linkFlags); unit += (lu.kind == mcpp::build::LinkUnit::TestBinary) ? flags.ldStdlibTest : flags.ldStdlibDefault; if (!unit.empty()) diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index 743af2c5..c58d728e 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -356,6 +356,32 @@ TEST(NinjaBackend, LinkFlagsAreNotReQuoted) { EXPECT_EQ(ninja.find("'\\''"), std::string::npos) << ninja; } +// Regression (#234 follow-up): a raw descriptor flag that packs two argv +// tokens into one string — e.g. compat.lua's `-include
` — must pass +// through VERBATIM so the shell splits it back into `-include` + the header. +// Blanket shell-quoting wrapped it into one malformed arg, so gcc looked for a +// file literally named "header" → "No such file" → aarch64/macos/windows +// cross-builds failed. Only `-D`/`/D` define tokens with an intra-value space +// get quoted; `-include foo.h` does not. +TEST(NinjaBackend, RawMultiTokenFlagIsNotQuoted) { + auto plan = minimal_plan(); + plan.compileUnits.push_back({ + .source = "src/main.c", + .object = "obj/main.o", + .packageName = "include_flag_test", + .packageCflags = {"-include mcpp_lua_platform_config.h"}, + }); + + auto ninja = emit_ninja_string(plan); + + // Verbatim, so the shell re-splits into two args. + EXPECT_NE(ninja.find("unit_cflags = -include mcpp_lua_platform_config.h"), + std::string::npos) << ninja; + // Must NOT be wrapped in quotes (which would make it one malformed arg). + EXPECT_EQ(ninja.find("'-include mcpp_lua_platform_config.h'"), + std::string::npos) << ninja; +} + TEST(NinjaBackend, RootPackageCxxflagsAreEmittedOncePerUnit) { auto plan = minimal_plan(); plan.manifest.buildConfig.cxxflags = {"-DROOT_FLAG=1"}; From 24fb8d4227a33115e84e3ae7ac5c736a1a6dd0fe Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 21:20:50 +0800 Subject: [PATCH 20/21] test(ninja): make 3 emission-format tests host-aware (fix windows CI) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit test_ninja_backend asserted Linux/POSIX-shaped output that fails on the windows self-host runner (unrelated to product behavior, which is correct per-platform): (1) CxxModule...Depfile asserts the POSIX-only filtered gcc depfile → skip on Windows (MSVC uses deps=msvc); (2) MsvcIncludeDirs... breaks on the runner's C:\ temp path whose ':' is ninja-escaped → run the (host- independent) MSVC-dialect check on POSIX; (3) QuotesFlagValueWithSpace expects single-quote but shell_quote_arg uses double-quote on Windows → assert the platform-appropriate spelling. Product unchanged. --- tests/unit/test_ninja_backend.cpp | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index c58d728e..364cb452 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -7,6 +7,7 @@ import mcpp.build.ninja; import mcpp.build.plan; import mcpp.manifest; import mcpp.toolchain.model; +import mcpp.platform; using namespace mcpp::build; @@ -82,6 +83,12 @@ TEST(NinjaBackend, ObjectiveCSourceUsesCObjectRuleAndCFlags) { // -MMD output that ninja's depfile loader rejects — see the long comment at // the definition site for the empirically-confirmed failure mode. TEST(NinjaBackend, CxxModuleAndCxxObjectRulesTrackHeaderDepsViaGccDepfile) { + // The filtered gcc depfile (#235) is POSIX-only: `posixDepfile = + // !msvcDeps && !is_windows` (awk isn't available on native Windows, and + // MSVC uses `deps = msvc` instead). This asserts the POSIX emission. + if constexpr (mcpp::platform::is_windows) + GTEST_SKIP() << "gcc depfile filter is POSIX-only (Windows uses deps=msvc)"; + auto plan = minimal_plan(); auto ninja = emit_ninja_string(plan); @@ -175,6 +182,13 @@ TEST(NinjaBackend, CxxFlagsIncludeBuildIncludeDirs) { // prepending the dialect prefix. This test would FAIL before the fix // (emitting the literal, unrewritten "/Iinclude") and passes after. TEST(NinjaBackend, MsvcIncludeDirsAreAbsolutizedNotGnuNormalized) { + // The MSVC-dialect logic under test is host-independent; run it on POSIX + // where the test's temp projectRoot has no drive letter. On Windows the + // runner's `C:\...` temp path gets its `:` ninja-escaped (`C$:`), which + // would need escape-aware matching unrelated to what this test verifies. + if constexpr (mcpp::platform::is_windows) + GTEST_SKIP() << "MSVC-dialect path check runs on POSIX (avoids Windows drive-colon ninja escaping)"; + auto plan = minimal_plan(); plan.toolchain.compiler = mcpp::toolchain::CompilerId::MSVC; plan.toolchain.binaryPath = "cl.exe"; @@ -300,8 +314,12 @@ TEST(NinjaBackend, QuotesFlagValueWithSpace) { auto ninja = emit_ninja_string(plan); - EXPECT_NE(ninja.find("unit_cflags = '-DT=long long'"), std::string::npos) - << ninja; + // shell_quote_arg wraps in single quotes on POSIX, double quotes on + // Windows — assert the platform-appropriate spelling. + const std::string quoted = mcpp::platform::is_windows + ? "unit_cflags = \"-DT=long long\"" + : "unit_cflags = '-DT=long long'"; + EXPECT_NE(ninja.find(quoted), std::string::npos) << ninja; // Must NOT appear as two bare, unquoted words split on the space. EXPECT_EQ(ninja.find("unit_cflags = -DT=long long"), std::string::npos) << ninja; From 78bb1c915e56a145a626db2240d8f4dd41a64a8c Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 18 Jul 2026 21:33:03 +0800 Subject: [PATCH 21/21] fix(build): scope #235 compile-edge depfile to GCC; e2e 118 requires gcc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The awk filter strips GCC -fmodules' reversed make-rules; Clang's module (.pcm) depfile shape differs and applying the GCC-shaped filter there is untested. macOS CI (Clang) failed e2e 118 because the purview-rebuild depfile didn't behave. Gate posixDepfile to CompilerId::GCC — Clang/MSVC keep the pre-#235 behavior (no compile-edge depfile; header/purview rebuild tracking on Clang is a follow-up, same as 0.0.96 → not a regression). e2e 118 now declares `# requires: gcc` so it skips where GCC isn't the toolchain (macOS has no gcc capability per run_all.sh). Linux GCC: depfile active, 117/118 green. --- src/build/ninja_backend.cppm | 14 +++++++++++++- tests/e2e/118_purview_include_rebuild.sh | 4 +++- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 73ab5419..e1a97dd5 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -456,7 +456,19 @@ std::string emit_ninja_string(const BuildPlan& plan) { // below), so a non-MSVC Windows toolchain keeps the pre-#235 // behavior (no depfile) rather than depend on an unavailable filter — // msvcDeps (cl.exe) is unaffected either way (deps=msvc, no -MMD). - const bool posixDepfile = !msvcDeps && !mcpp::platform::is_windows; + // + // GCC-only: the awk filter strips the "reversed" make-rules that GCC's + // `-fmodules -MMD` bolts onto a module-TU depfile (`gcm.cache/.gcm: | + // ` etc., which ninja rejects as "inputs may not also have inputs"). + // Clang's module system (.pcm) does not emit those and its module-TU + // depfile shape is different; applying the GCC-shaped filter there is + // untested and could yield a wrong dep set. Until Clang is verified, + // scope this to GCC — Clang keeps the pre-#235 behavior (no compile-edge + // depfile; header/purview rebuild tracking on Clang is a follow-up, same + // as it was on 0.0.96, so this is not a regression). e2e 118 declares + // `# requires: gcc` and skips where GCC is not the toolchain. + const bool posixDepfile = !msvcDeps && !mcpp::platform::is_windows + && plan.toolchain.compiler == mcpp::toolchain::CompilerId::GCC; const std::string mmd_flag = posixDepfile ? "-MMD -MF $out.d.raw " : ""; const std::string mmd_filter = posixDepfile ? " && awk 'NR==1{print;next} /^[^ ]/{exit} {print}' " diff --git a/tests/e2e/118_purview_include_rebuild.sh b/tests/e2e/118_purview_include_rebuild.sh index ff0eccbe..efb73b9e 100755 --- a/tests/e2e/118_purview_include_rebuild.sh +++ b/tests/e2e/118_purview_include_rebuild.sh @@ -1,6 +1,8 @@ #!/usr/bin/env bash -# requires: +# requires: gcc # mcpp#235: compile edges must track header/purview/GMF `#include`s via a +# GCC-only depfile (the awk filter strips GCC -fmodules' reversed rules; +# Clang/MSVC keep pre-#235 behavior — see ninja_backend posixDepfile gate). # depfile. Before this fix, `cxx_module`/`cxx_object` had NO depfile on # non-MSVC (only the msvcDeps branch added `deps=msvc`) — the P1689 scan's # `$out.dep` was generated then discarded. So editing a file `#include`d