Skip to content

test(e2e-helper): verify + retry remote-scope registration in capsule builds#10485

Open
davidfirst wants to merge 2 commits into
masterfrom
harden-e2e-remote-registration
Open

test(e2e-helper): verify + retry remote-scope registration in capsule builds#10485
davidfirst wants to merge 2 commits into
masterfrom
harden-e2e-remote-registration

Conversation

@davidfirst

Copy link
Copy Markdown
Member

Problem

Integration specs that create file-based remote scopes (snapping/checkout/lanes via mockWorkspace) intermittently fail only under bit ci pr --build (capsule builds), not under bit test, with:

  • InvalidScopeNameFromRemote: cannot find scope '<hash>-remote'
  • MissingBitMapComponent: component "<hash>-remote/comp1" was not found

Root cause: these in-process integration tests share global state with no per-test isolation (the global config/scope dir is frozen at module-load from ~/Library/Caches/Bit). Under a build, dozens of them run back-to-back in one long-lived process alongside the host build machinery, so a just-registered <hash>-remote occasionally isn't resolvable to a later import, which then falls through to the bit.cloud hub.

This change

Hardens the e2e-helper remote registration as a low-risk first step:

  • ScopeHelper.addRemoteScope() now verifies the remote actually landed in the workspace scope.json and re-adds it (bounded retry) if not — recovering a transient write race. If it still can't be verified it logs a loud, diagnosable message (does not throw), so the hundreds of other callers can't regress and any residual CI failure clearly points at registration vs. an import-side resolution problem.
  • mockWorkspace({ bareScopeName }) warns clearly if the reconstructed bare-scope path doesn't exist, instead of surfacing a cryptic scope-resolution error later.

Test-infra only; no product behavior change. This is deliberately conservative — we'll watch several bit_pr runs to see whether it removes the flake or localizes it to the import side for a follow-up.

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Harden e2e remote-scope registration with verification + bounded retry

🐞 Bug fix 🧪 Tests 🕐 10-20 Minutes

Grey Divider

AI Description

• Verify file-based remotes are actually registered in workspace scope.json after `bit remote
 add`.
• Retry remote registration a few times to reduce capsule-build flakiness.
• Add clear diagnostics when reconstructed bare-scope paths are missing in mockWorkspace.
Diagram

graph TD
  A["Integration specs"] --> B["mockWorkspace()"] --> C["ScopeHelper.addRemoteScope()"] --> D["bit remote add (file://)"] --> E[(".bit/scope.json")]
  C --> F{"Remote registered?"}
  F -->|"No (up to 3 retries)"| D
  F -->|"Still no"| G["Log warning"]
  B --> H{"Bare scope path exists?"}
  H -->|"No"| I["Log warning"]

  subgraph Legend
    direction LR
    _proc["Process/step"] ~~~ _file[("File")]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Isolate per-test global Bit state (temp home/config)
  • ➕ Addresses the root cause (shared global config/scope across many specs)
  • ➕ Reduces reliance on retries/time-based mitigation
  • ➕ Improves determinism across all integration specs
  • ➖ Broader refactor touching many tests/helpers
  • ➖ Higher risk of breaking assumptions in existing test infrastructure
  • ➖ More time-consuming to implement and validate
2. Run integration specs in separate processes (strong isolation)
  • ➕ Eliminates long-lived-process state bleed in capsule builds
  • ➕ Makes failures more attributable to the spec itself
  • ➖ Higher CI runtime and resource usage
  • ➖ More complex orchestration and potential flake from process management
3. Make remote-add atomic/observable via CLI output or API (no file polling)
  • ➕ Avoids ad-hoc filesystem verification logic
  • ➕ Can provide structured diagnostics and deterministic success/failure
  • ➖ May require changes in Bit CLI/core behavior
  • ➖ Potentially larger scope than desired for a conservative test-only fix

Recommendation: Given the goal of a low-risk, test-infra-only mitigation, the chosen approach (verify .bit/scope.json + bounded retry + loud warnings) is appropriate as a first step: it reduces transient registration races without breaking the many existing callers. If flakiness persists, prioritize isolating global Bit state per test/capsule run, since that targets the underlying shared-state issue rather than symptoms.

Files changed (2) +60 / -1

Bug fix (1) +49 / -1
e2e-scope-helper.tsVerify and retry file-remote registration via workspace scope.json +49/-1

Verify and retry file-remote registration via workspace scope.json

• After running 'bit remote add file://...', the helper now checks whether the remote was persisted into '<cwd>/.bit/scope.json'. For non-global remotes it performs a bounded retry loop and logs a detailed warning (including current remotes) if registration still cannot be verified.

components/legacy/e2e-helper/e2e-scope-helper.ts

Tests (1) +11 / -0
mock-workspace.tsWarn when reconstructed bare-scope path is missing in second-workspace flow +11/-0

Warn when reconstructed bare-scope path is missing in second-workspace flow

• When 'mockWorkspace({ bareScopeName })' reconstructs a remote path from the provided scope name, it now emits a clear warning if that bare scope directory does not exist. This helps distinguish cleanup/ordering issues from later import-time scope resolution errors.

scopes/workspace/testing/mock-workspace/mock-workspace.ts

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Jul 10, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (1) 📜 Skill insights (0)

Grey Divider


Action required

1. Bare-scope verification broken ✓ Resolved 🐞 Bug ≡ Correctness
Description
ScopeHelper.addRemoteScope() verifies remotes by reading /.bit/scope.json, but the helper is
also used with cwd set to a bare scope root where the file is /scope.json. This makes
verification always fail for those call sites, causing redundant retries and warning spam (and
unnecessary runtime) even when registration succeeded.
Code

components/legacy/e2e-helper/e2e-scope-helper.ts[R127-165]

+    if (!isGlobal) {
+      const maxRetries = 3;
+      for (let attempt = 0; attempt < maxRetries && !this.isRemoteRegistered(remoteScopePath, cwd); attempt += 1) {
+        this.command.runCmd(cmd, cwd);
+      }
+      if (!this.isRemoteRegistered(remoteScopePath, cwd)) {
+        // eslint-disable-next-line no-console
+        console.log(
+          `[e2e-helper] WARNING: remote "file://${remoteScopePath}" is not registered in the workspace ` +
+            `scope.json at "${cwd}" after ${maxRetries + 1} attempts. remotes on disk: ` +
+            JSON.stringify(this.readWorkspaceRemotes(cwd))
+        );
+      }
+    }
+    return result;
+  }
+
+  /**
+   * read the `remotes` map from the workspace scope.json (`<cwd>/.bit/scope.json`). the map is
+   * `{ [scopeName]: host }` (see ScopeJson.addRemote), and for e2e file remotes the scope name is
+   * the remote directory's basename.
+   */
+  private readWorkspaceRemotes(cwd: string): Record<string, string> {
+    const scopeJsonPath = path.join(cwd, '.bit', 'scope.json');
+    if (!fs.existsSync(scopeJsonPath)) return {};
+    const scopeJson = fs.readJsonSync(scopeJsonPath, { throws: false });
+    return (scopeJson && scopeJson.remotes) || {};
+  }
+
+  private isRemoteRegistered(remoteScopePath: string, cwd: string): boolean {
+    const scopeName = path.basename(remoteScopePath);
+    const remotes = this.readWorkspaceRemotes(cwd);
+    // match by scope name (the registered key) or by the host value referencing the remote path —
+    // basename is stable against any path normalization the remote-add may apply to the host.
+    return (
+      scopeName in remotes ||
+      Object.values(remotes).some((host) => typeof host === 'string' && host.includes(scopeName))
+    );
}
Evidence
The new verification reads only /.bit/scope.json, but addRemoteScope() is used with cwd set to
a bare scope root; bare scopes store scope.json directly under the scope root (/scope.json).
Therefore, for those call sites the verification will always read {} and never observe the
newly-added remote, triggering retries/warnings every time.

components/legacy/e2e-helper/e2e-scope-helper.ts[110-165]
e2e/harmony/isolate-cyclic-deps.e2e.ts[82-87]
components/legacy/scope/scope-json.ts[13-15]
components/legacy/scope/scope.ts[726-754]
components/legacy/scope/scope-loader.ts[7-10]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ScopeHelper.addRemoteScope()` introduced a verification+retry loop that calls `readWorkspaceRemotes(cwd)`, which hardcodes the workspace layout (`<cwd>/.bit/scope.json`). However, many tests call `addRemoteScope(remotePath, cwd)` with `cwd` set to a **bare scope** directory (created by `bit init --bare`), where `scope.json` is located at `<cwd>/scope.json`.
As a result, verification always returns `{}` for bare scopes, so `isRemoteRegistered()` stays `false`, triggering the full retry loop and logging warnings even when the remote was actually added correctly.
### Issue Context
- `addRemoteScope()` is invoked with `cwd` pointing to a bare scope in multiple e2e flows (e.g. `helper.scopeHelper.addRemoteScope(helper.scopes.remotePath, bareScope.scopePath)`), not only workspaces.
- Core scope code computes the scope.json path as `path.join(scopePath, SCOPE_JSON)` (no `.bit/`) for a scope root.
### Fix Focus Areas
- components/legacy/e2e-helper/e2e-scope-helper.ts[127-165]
### Suggested fix approach
1. Make `readWorkspaceRemotes()` able to read remotes from **either**:
- workspace scope: `<cwd>/.bit/scope.json` (workspace root)
- bare-scope scope: `<cwd>/scope.json` (bare scope root)
- (optional) or derive the actual scope path via `findScopePath(cwd)` and then read `<scopePath>/scope.json`.
2. Consider only enabling the retry/verification logic when the resolved scope.json path exists (and is the one the `bit remote add` command will write to for that `cwd`).
3. Update the warning message to print the *actual* scope.json path used for verification to make diagnostics accurate.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Raw console.log warnings added 📘 Rule violation ⚙ Maintainability ⭐ New
Description
New warning output is emitted via direct console.log with hardcoded text, rather than using the
repository’s shared CLI output formatter/style guide. This can lead to inconsistent CLI/test-run
output and makes future output changes harder to standardize.
Code

components/legacy/e2e-helper/e2e-scope-helper.ts[R133-138]

+        // eslint-disable-next-line no-console
+        console.log(
+          `[e2e-helper] WARNING: remote "file://${remoteScopePath}" is not registered in the workspace ` +
+            `scope.json at "${cwd}" after ${maxRetries + 1} attempts. remotes on disk: ` +
+            JSON.stringify(this.readWorkspaceRemotes(cwd))
+        );
Evidence
PR Compliance ID 1 requires CLI/output text changes to follow the CLI output style guide and use the
shared formatter from @teambit/cli, avoiding ad-hoc/hardcoded output. The PR adds direct
console.log(...) warning strings in two places, which is inconsistent with that requirement.

CLAUDE.md: CLI Output Must Follow Repository Style Guide and Use Shared Output Formatter (No Hardcoded Chalk Styles/Symbols)
components/legacy/e2e-helper/e2e-scope-helper.ts[133-138]
scopes/workspace/testing/mock-workspace/mock-workspace.ts[28-31]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
New warning messages are printed with direct `console.log` and hardcoded formatting, instead of using the shared CLI output formatter per the repository style guide.

## Issue Context
This PR introduces new diagnostic output in test infrastructure (`e2e-helper` and `mockWorkspace`). Even if intended for CI diagnostics, the repository compliance rule requires output changes to follow the shared formatting toolkit and style guide to keep output consistent.

## Fix Focus Areas
- components/legacy/e2e-helper/e2e-scope-helper.ts[133-138]
- scopes/workspace/testing/mock-workspace/mock-workspace.ts[28-31]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Remote check too permissive 🐞 Bug ☼ Reliability ⭐ New
Description
ScopeHelper.isRemoteRegistered() treats a remote as registered if any existing remote host string
merely contains the new remote directory basename, which can match unrelated remotes and incorrectly
suppress the retry/warning path. This reduces the effectiveness/diagnostic value of the new
hardening in flaky CI runs.
Code

components/legacy/e2e-helper/e2e-scope-helper.ts[R160-168]

+  private isRemoteRegistered(remoteScopePath: string, cwd: string): boolean {
+    const scopeName = path.basename(remoteScopePath);
+    const remotes = this.readWorkspaceRemotes(cwd);
+    // match by scope name (the registered key) or by the host value referencing the remote path —
+    // basename is stable against any path normalization the remote-add may apply to the host.
+    return (
+      scopeName in remotes ||
+      Object.values(remotes).some((host) => typeof host === 'string' && host.includes(scopeName))
+    );
Evidence
The CLI implementation stores the remote under remote.name and writes the host as remote.host,
so verification should target that deterministic mapping; substring matching on host can incorrectly
report success when a different remote’s host happens to contain the same substring.

components/legacy/e2e-helper/e2e-scope-helper.ts[160-168]
scopes/harmony/global-config/remote.ts[9-28]
components/legacy/scope/scope-json.ts[110-113]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`isRemoteRegistered()` currently returns true when **any** configured remote host string `includes(scopeName)`. This can yield false positives (substring matches) and skip the retry/warning branch even when the intended remote wasn’t actually registered.

## Issue Context
`bit remote add` persists remotes under a deterministic key (`remote.name`) and value (`remote.host`). The verification can be made precise by checking the expected key and/or normalizing the configured host into a path/URL and comparing it to the intended `remoteScopePath`.

## Fix Focus Areas
- components/legacy/e2e-helper/e2e-scope-helper.ts[160-168]

### Suggested fix approach
- Prefer verifying by **key** (remote scope name) and **exact host** rather than substring:
 - Determine the expected remote name:
   - Option A (best): read `${remoteScopePath}/scope.json` and use its `name` field.
   - Option B: keep `path.basename(remoteScopePath)` but then do *not* use substring matching.
 - Determine the expected host for file remotes:
   - Compare against the exact URL you pass to `bit remote add` (or normalize both to filesystem paths using `fileURLToPath` when the host is a `file://` URL).
- Replace `host.includes(scopeName)` with either:
 - an exact key check (`expectedName in remotes`), and/or
 - a precise host comparison for that key (`remotes[expectedName] === expectedHost` or normalized path equality).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Previous review results

Review updated until commit b5f8bf4

Results up to commit eabb731


🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)


Action required
1. Bare-scope verification broken 🐞 Bug ≡ Correctness
Description
ScopeHelper.addRemoteScope() verifies remotes by reading <cwd>/.bit/scope.json, but the helper
is also used with cwd set to a bare scope root where the file is <cwd>/scope.json. This makes
verification always fail for those call sites, causing redundant retries and warning spam (and
unnecessary runtime) even when registration succeeded.
Code

components/legacy/e2e-helper/e2e-scope-helper.ts[R127-165]

+    if (!isGlobal) {
+      const maxRetries = 3;
+      for (let attempt = 0; attempt < maxRetries && !this.isRemoteRegistered(remoteScopePath, cwd); attempt += 1) {
+        this.command.runCmd(cmd, cwd);
+      }
+      if (!this.isRemoteRegistered(remoteScopePath, cwd)) {
+        // eslint-disable-next-line no-console
+        console.log(
+          `[e2e-helper] WARNING: remote "file://${remoteScopePath}" is not registered in the workspace ` +
+            `scope.json at "${cwd}" after ${maxRetries + 1} attempts. remotes on disk: ` +
+            JSON.stringify(this.readWorkspaceRemotes(cwd))
+        );
+      }
+    }
+    return result;
+  }
+
+  /**
+   * read the `remotes` map from the workspace scope.json (`<cwd>/.bit/scope.json`). the map is
+   * `{ [scopeName]: host }` (see ScopeJson.addRemote), and for e2e file remotes the scope name is
+   * the remote directory's basename.
+   */
+  private readWorkspaceRemotes(cwd: string): Record<string, string> {
+    const scopeJsonPath = path.join(cwd, '.bit', 'scope.json');
+    if (!fs.existsSync(scopeJsonPath)) return {};
+    const scopeJson = fs.readJsonSync(scopeJsonPath, { throws: false });
+    return (scopeJson && scopeJson.remotes) || {};
+  }
+
+  private isRemoteRegistered(remoteScopePath: string, cwd: string): boolean {
+    const scopeName = path.basename(remoteScopePath);
+    const remotes = this.readWorkspaceRemotes(cwd);
+    // match by scope name (the registered key) or by the host value referencing the remote path —
+    // basename is stable against any path normalization the remote-add may apply to the host.
+    return (
+      scopeName in remotes ||
+      Object.values(remotes).some((host) => typeof host === 'string' && host.includes(scopeName))
+    );
  }
Evidence
The new verification reads only <cwd>/.bit/scope.json, but addRemoteScope() is used with cwd
set to a bare scope root; bare scopes store scope.json directly under the scope root
(<scopePath>/scope.json). Therefore, for those call sites the verification will always read {}
and never observe the newly-added remote, triggering retries/warnings every time.

components/legacy/e2e-helper/e2e-scope-helper.ts[110-165]
e2e/harmony/isolate-cyclic-deps.e2e.ts[82-87]
components/legacy/scope/scope-json.ts[13-15]
components/legacy/scope/scope.ts[726-754]
components/legacy/scope/scope-loader.ts[7-10]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`ScopeHelper.addRemoteScope()` introduced a verification+retry loop that calls `readWorkspaceRemotes(cwd)`, which hardcodes the workspace layout (`<cwd>/.bit/scope.json`). However, many tests call `addRemoteScope(remotePath, cwd)` with `cwd` set to a **bare scope** directory (created by `bit init --bare`), where `scope.json` is located at `<cwd>/scope.json`.

As a result, verification always returns `{}` for bare scopes, so `isRemoteRegistered()` stays `false`, triggering the full retry loop and logging warnings even when the remote was actually added correctly.

### Issue Context
- `addRemoteScope()` is invoked with `cwd` pointing to a bare scope in multiple e2e flows (e.g. `helper.scopeHelper.addRemoteScope(helper.scopes.remotePath, bareScope.scopePath)`), not only workspaces.
- Core scope code computes the scope.json path as `path.join(scopePath, SCOPE_JSON)` (no `.bit/`) for a scope root.

### Fix Focus Areas
- components/legacy/e2e-helper/e2e-scope-helper.ts[127-165]

### Suggested fix approach
1. Make `readWorkspaceRemotes()` able to read remotes from **either**:
  - workspace scope: `<cwd>/.bit/scope.json` (workspace root)
  - bare-scope scope: `<cwd>/scope.json` (bare scope root)
  - (optional) or derive the actual scope path via `findScopePath(cwd)` and then read `<scopePath>/scope.json`.
2. Consider only enabling the retry/verification logic when the resolved scope.json path exists (and is the one the `bit remote add` command will write to for that `cwd`).
3. Update the warning message to print the *actual* scope.json path used for verification to make diagnostics accurate.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Qodo Logo

Comment thread components/legacy/e2e-helper/e2e-scope-helper.ts
Comment on lines +133 to +138
// eslint-disable-next-line no-console
console.log(
`[e2e-helper] WARNING: remote "file://${remoteScopePath}" is not registered in the workspace ` +
`scope.json at "${cwd}" after ${maxRetries + 1} attempts. remotes on disk: ` +
JSON.stringify(this.readWorkspaceRemotes(cwd))
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. Raw console.log warnings added 📘 Rule violation ⚙ Maintainability

New warning output is emitted via direct console.log with hardcoded text, rather than using the
repository’s shared CLI output formatter/style guide. This can lead to inconsistent CLI/test-run
output and makes future output changes harder to standardize.
Agent Prompt
## Issue description
New warning messages are printed with direct `console.log` and hardcoded formatting, instead of using the shared CLI output formatter per the repository style guide.

## Issue Context
This PR introduces new diagnostic output in test infrastructure (`e2e-helper` and `mockWorkspace`). Even if intended for CI diagnostics, the repository compliance rule requires output changes to follow the shared formatting toolkit and style guide to keep output consistent.

## Fix Focus Areas
- components/legacy/e2e-helper/e2e-scope-helper.ts[133-138]
- scopes/workspace/testing/mock-workspace/mock-workspace.ts[28-31]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +160 to +168
private isRemoteRegistered(remoteScopePath: string, cwd: string): boolean {
const scopeName = path.basename(remoteScopePath);
const remotes = this.readWorkspaceRemotes(cwd);
// match by scope name (the registered key) or by the host value referencing the remote path —
// basename is stable against any path normalization the remote-add may apply to the host.
return (
scopeName in remotes ||
Object.values(remotes).some((host) => typeof host === 'string' && host.includes(scopeName))
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. Remote check too permissive 🐞 Bug ☼ Reliability

ScopeHelper.isRemoteRegistered() treats a remote as registered if any existing remote host string
merely contains the new remote directory basename, which can match unrelated remotes and incorrectly
suppress the retry/warning path. This reduces the effectiveness/diagnostic value of the new
hardening in flaky CI runs.
Agent Prompt
## Issue description
`isRemoteRegistered()` currently returns true when **any** configured remote host string `includes(scopeName)`. This can yield false positives (substring matches) and skip the retry/warning branch even when the intended remote wasn’t actually registered.

## Issue Context
`bit remote add` persists remotes under a deterministic key (`remote.name`) and value (`remote.host`). The verification can be made precise by checking the expected key and/or normalizing the configured host into a path/URL and comparing it to the intended `remoteScopePath`.

## Fix Focus Areas
- components/legacy/e2e-helper/e2e-scope-helper.ts[160-168]

### Suggested fix approach
- Prefer verifying by **key** (remote scope name) and **exact host** rather than substring:
  - Determine the expected remote name:
    - Option A (best): read `${remoteScopePath}/scope.json` and use its `name` field.
    - Option B: keep `path.basename(remoteScopePath)` but then do *not* use substring matching.
  - Determine the expected host for file remotes:
    - Compare against the exact URL you pass to `bit remote add` (or normalize both to filesystem paths using `fileURLToPath` when the host is a `file://` URL).
- Replace `host.includes(scopeName)` with either:
  - an exact key check (`expectedName in remotes`), and/or
  - a precise host comparison for that key (`remotes[expectedName] === expectedHost` or normalized path equality).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit b5f8bf4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant