ci(1606): standard-checks and helm-ci call the Makefile instead of copying it - #723
Merged
Conversation
* fix(resource-monitor): always grant read-only ClusterRole (decouple from clusterScope)
Under clusterScope: false the chart rendered only a namespace-scoped Role in
the release namespace. But the resource-monitor's code:
* calls core_v1_api.list_pod_for_all_namespaces(field_selector=spec.nodeName=...)
-- a CLUSTER-SCOPED list verb a namespaced Role can never satisfy; and
* read_namespaced_pod()s its OWN pod, which lives in
.Values.nodeAgents.namespace.name (NOT .Release.Namespace).
So with clusterScope: false the DaemonSet 403'd on startup and crashlooped
(70+ restarts observed on a live cluster). Per-node monitoring is intrinsically
cluster-scoped.
Always render the read-only ClusterRole + ClusterRoleBinding regardless of
clusterScope (get/list/watch on pods/nodes/namespaces + metrics; no write,
exec, or secret access). resourceMonitor: false still fully disables the
component. clusterScope continues to gate the training/jobs isolation footprint
elsewhere -- it must not leave the node monitor without permissions it cannot
run without.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(resource-monitor): assert always-cluster-scoped RBAC under clusterScope=false
Follow-up to the RBAC fix: node_agents_namespace_test.yaml still asserted the
old behavior (namespaced Role + RoleBinding in the release namespace when
clusterScope=false). Update that case to assert the corrected contract -- a
ClusterRole + ClusterRoleBinding always render (with no metadata.namespace),
while the subject SA still lives in the node-agents namespace.
The clusterScope=false path stays under test; only the asserted behavior
changes to match the fix. Verified with `helm unittest` (all resource-monitor
suites pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(rbac): grant `get` on configmaps/secrets to jobs-manager SA
The ingestion endpoint's orphan-resource verify path (client-runtime#52)
and missing-row self-heal (client-runtime#54) read the existing
ConfigMap/Secret on a create-409 to confirm content matches before
reuse. The Role/ClusterRole only granted `create`, so those reads
returned Forbidden and the endpoint 500'd instead of the intended
409/200-replay — verified live on the dev cluster.
Add `get` alongside `create` in both the ClusterRole (clusterScope:
true) and namespace Role (clusterScope: false) branches.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci(helm): guard that the pinned ingestor digest is multi-arch (closes #186) (#187)
Add a helm-ci job (ingestor-multiarch) that parses images.ingestor.digest and
fails the build unless it's a multi-arch index (linux/amd64 + linux/arm64).
Greenfield installs spawn the ingestor Job from this PINNED digest before
image-refresh first ticks, so an amd64-only pin breaks data ingestion on arm64
hosts (Apple Silicon, Graviton) with "no match for platform" / ImagePullBackOff.
This would have caught #160 (the amd64-only v0.3.1 pin) before it shipped.
ghcr.io/tracebloc/ingestor is public -> no secrets. Verified: passes on the
current multi-arch baseline (sha256:d361fa77, v0.3.2 / #184), fails on the old
amd64-only sha256:a0861ea9.
Note: the digest is already multi-arch on develop as of v0.3.2 (#184 — the same
d361fa77 index this PR previously bumped to), so #187 no longer touches
values.yaml; it adds only the regression guard so an amd64-only pin can't slip
back in.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(#190): fail image-refresh loudly when the ingestor (ghcr) digest can't resolve (#191)
image-refresh silently skipped every tick when get_latest_digest returned
empty for the ghcr.io ingestor image (egress/proxy/firewall to ghcr.io, or a
blocked token endpoint) — never reaching the registry-drift branch that sets
the new digest. jobs-manager + pods-monitor pull from docker.io and refreshed
fine, so the CronJob looked healthy while the ingestor digest stayed pinned on
the install-time baseline. That's why the berlin-team arm64 install sat on the
amd64-only v0.3.1 digest even after :0.3 went multi-arch (#186 follow-up #2).
Now count consecutive ingestor-resolve failures on a deployment annotation:
- below imageRefresh.ingestorResolveFailureThreshold (default 3, ~45 min at the
15-min schedule) -> WARN + skip, as before (tolerate transient blips);
- at/above it -> ERROR with actionable guidance, a
tracebloc.io/ingestor-refresh-last-error annotation, and a non-zero exit so
the Job fails visibly in `kubectl get cronjob` / monitoring — the same
surfacing idiom Pass 2's stuck-rollout check already relies on;
- a successful resolve clears the streak.
Threshold is nil-guarded (default 3) for --reuse-values upgrades and
schema-validated (integer >= 1). The digest-resolution logic itself is
unchanged (verified correct: it returns the multi-arch index digest).
helm unittest 146/146, helm lint clean, shellcheck + sh -n clean.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* test(requests-proxy): add helm-unittest coverage for requests-proxy Deployment (#194)
requests-proxy-deployment.yaml was the only data-plane workload template
without a unit test. This suite pins the properties most costly to regress:
- security-context invariants (no SA-token automount, runAsNonRoot,
seccomp RuntimeDefault, runAsUser 1001, no privilege escalation,
drop ALL caps, read-only root filesystem) — see docs/SECURITY.md
- the single-replica / single gunicorn worker constraint (the pod token
registry is process-local; >1 worker silently shards token lookups)
- the docker.io/tracebloc/jobs-manager image source and port 8888
- the nil-guarded resource defaults, plus an override case that exercises
the default-through-dict fallthrough (guards the historic
`readOnlyRootFilesystem: trueresources:` newline-eating regression)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(#196): allow training-pod egress to the requests-proxy (8888) (#197)
The training-egress NetworkPolicy denies all pod-to-pod / ClusterIP
egress (rule 2 excepts the cluster CIDRs) and re-permits only MySQL
(rule 3). When the requests-proxy architecture shipped — training pods
POST epoch results / FLOPs to requests-proxy-service:8888 instead of
holding Service Bus credentials — this template was never updated to
re-permit egress to the proxy. Result on every install with the policy
enabled: pods hit "requests-proxy-service:8888 ... [Errno 111]
Connection refused" at the first epoch finalize → CrashLoopBackOff →
all experiments fail.
Add rule 4 mirroring the MySQL rule: TCP/8888 to podSelector
app=requests-proxy (same namespace). Service selector + port from
templates/requests-proxy-service.yaml.
Verified: `helm template -f ci/bm-values.yaml --show-only
templates/network-policy-training.yaml` renders the new rule as valid
YAML.
Found live on a fresh client (tracebloc-amazon / k3d): jobs-manager
reached the proxy (HTTP 401) while training pods got connection-refused
— the only differentiator was this egress policy. Interim: live-patched
the cluster + suspended its auto-upgrade CronJob (so reuse-values
wouldn't revert the patch); re-enable once this lands + releases.
Closes #196.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Installer UX: drop PriorityClass, fix namespace, one-per-machine guard, surface version (#192)
* fix(chart): drop the data-plane PriorityClass by default
The cluster-scoped, fixed-name `tracebloc-data-plane` PriorityClass was the
only thing forcing one tracebloc client per cluster (a second release collided
on it with a cryptic Helm error) and blocking multiple tracebloc namespaces in
one BYO cluster. mysql doesn't need it: memory requests==limits (last evicted
under memory pressure), data on a PVC (eviction = transient restart, not data
loss), and a PDB guards voluntary disruptions. Its only unique benefit was
letting the scheduler preempt training jobs to keep mysql scheduled on a packed
node — a narrow case.
Default priorityClass.create=false + name="" so new installs template no
PriorityClass and mysql carries no priorityClassName. Opt back in
(create:true + name) on contended clusters, or reference an out-of-band one
(create:false + name:<existing>). helm-unittest updated; 144/144 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(installer): fixed namespace + one-per-machine guard; drop workspace prompt
The "Choose a workspace name" prompt asked the user to invent a label that
isn't their identity (the backend identifies a client by its credentials, not
this string — it's just the local k8s namespace / Helm release name; the
installer even discards the auth response body). It defaulted to a meaningless
"default" and was the field that collided on a second install.
- Drop the prompt; TB_NAMESPACE defaults to a fixed "tracebloc"
(env-overridable for advanced/GitOps setups).
- One-client-per-machine guard: after credentials verify, compare the entered
Client ID against any client already installed here (helm get values). Same
ID = a normal re-run/upgrade; a DIFFERENT ID hard-blocks with an explanation
and options (repair / switch via `k3d cluster delete` / use another machine)
instead of silently re-pointing the machine. This replaces the accidental
PriorityClass collision (now dropped) with an intentional, explained guard.
- Document the TB_NAMESPACE override; update bats (input sequences + 2 new
guard tests). bats 26/27 — the 1 failure is a pre-existing macOS-bash-3.2
quirk in _extract_yaml_value, unrelated (CI bash 5 passes it).
NOTE: install-k8s.ps1 + its Pester tests still need the same mirror (follow-up).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(installer-ps): mirror fixed namespace + one-per-machine guard (PowerShell)
Mirrors the bash change in install-k8s.ps1:
- drop the "Choose a workspace name" prompt; TB_NAMESPACE defaults to a fixed
"tracebloc" (override via $env:TB_NAMESPACE).
- one-client-per-machine guard: after credentials verify, compare the entered
Client ID against any client already installed here (helm get values); a
different ID hard-blocks with the same explanation/options as bash.
- Pester: 2 new guard tests (block-different / allow-same). The existing
Install-ClientHelm tests use dispatch-by-prompt Read-Host mocks, so the
prompt removal doesn't disturb them.
No pwsh locally -> verified via CI (Pester ubuntu+windows + PSScriptAnalyzer).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(installer): scan all namespaces in the one-per-machine guard
The guard checked only the `tracebloc` namespace, so a client installed by an
older installer version (default namespace `default`, or a custom name) wasn't
detected -- a re-run could create a second coexisting client. Now enumerate all
client-chart releases (helm list -A) and compare each one's clientId, covering
both fresh and migrated installs. bash uses jq (already a dependency; falls
back to the tracebloc namespace if absent); PowerShell uses ConvertFrom-Json.
The block message names the namespace. bats + Pester guard tests updated.
Verified: bats green locally (jq path); PowerShell via CI Pester.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(installer): show client (chart) version in summary + --diagnose
Users had no easy way to see which client version they're on (the CLI isn't
shipped yet; `helm list` needs the namespace, and nothing surfaced it). Show
the chart version where they already look:
- install summary: a "Version" line next to Workspace.
- --diagnose: as the first console line + recorded in the bundle header
(the #1 thing support needs).
Adds a best-effort `_chart_version` / Get-ChartVersion helper (greps helm's
CHART column -> no jq). bash + PowerShell; bats + Pester coverage added.
Verified: summary.bats + diagnose.bats green locally; ps1 via CI Pester.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* chore: bump chart 1.4.4 → 1.4.5 to ship the training-egress proxy fix (#198)
1.4.4 is already published on the tracebloc.github.io/client Pages
channel and is what clusters run. The training-egress NetworkPolicy fix
(#197, allow training → requests-proxy:8888) merged to develop without a
version bump, so it is currently undeliverable: chart-releaser won't
overwrite the existing 1.4.4 release, and clusters already on 1.4.4 would
see no version change and pull nothing.
Bump to 1.4.5 (lockstep version/appVersion, matching 1.4.3/1.4.4 history)
so a v1.4.5 release publishes a new version that auto-upgrade actually
pulls. Chart-only change; no image change.
Ref #196 / #197.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: shujaat hasan <shujaathasan@shujaats-MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: shujaat_tracebloc <153823837+shujaatTracebloc@users.noreply.github.com>
Co-authored-by: lukasWuttke <54042461+LukasWodka@users.noreply.github.com>
…er (#203) feat(installer): install the tracebloc CLI as part of the client install (Step 5, both OSes)
…206) feat(secrets): wire POD_TOKEN_SIGNING_SECRET for stateless pod tokens (client-runtime#79)
Prod: Set kubeconfig context namespace after Helm install for tracebloc
release: promote develop → main (installer hardening — OOM gate, apt/needrestart fixes, drift CI)
Sync develop → main for v1.6.0 chart release
Sync develop → main for v1.6.1 chart release
feat(egress-proxy): training-pod egress lockdown — squid gateway, gated rollout (client-runtime#102)
…light helm test, inert) (#254) * test(charts): add helm-unittest suite for mysql-storage-pvc template (#250) The mysql-storage-pvc template (PVC + bare-metal hostPath PV backing the per-cluster MySQL state store) had no dedicated suite. Add one covering: - dynamic-PVC-only path (hostPath.enabled=false, managed default) - hostPath PV+PVC pair, claimRef binding, fixed release-scoped path - the helm.sh/resource-policy:keep annotation protecting the state store - access-mode defaulting, pvc size, and storageClass wiring Tests-only; no source/template/values changes. Security invariants unchanged. Co-authored-by: Claude <noreply@anthropic.com> * test(charts): add helm-unittest suite for logs-pvc template (#251) The logs-pvc template (PV + PVC for client logs) had no dedicated helm-unittest suite. Adds tests/logs_pvc_test.yaml covering the dynamic-provisioning PVC path and the hostPath PV+PVC path. Co-authored-by: Claude <noreply@anthropic.com> * Merge pull request #253 from tracebloc/feat/104-egress-enforcement-check feat(egress-proxy): deploy-time egress-enforcement pre-flight (non-blocking) [client-runtime#104] * fix(egress-proxy): enforcement check probes TCP reachability, not HTTP code [#104] (#255) Bugbot (PR #254): the check treated curl HTTP code 000 as proof of a block, but a TCP connection that succeeds (egress OPEN) then fails TLS verification also yields 000 — so `helm test` could pass on a non-enforcing CNI when probing an IP whose cert doesn't validate (the default 1.1.1.1 without -k). It conflated "TLS failed" with "egress blocked", defeating the check. Key the verdict off TCP reachability via curl's exit code instead of the HTTP status, and add -k so TLS is explicitly irrelevant: exit 28 (timeout) / 7 (connect failed) => egress blocked => test PASSES (exit 0); any other outcome (0 success, or a TLS-/HTTP-layer error such as 35/52/60 — all of which require the TCP connect to have already succeeded) => egress reached => NOT enforced => test FAILS (exit 1). Tests assert the verdict keys on the exit code (rc=$?, -k) and guard against a regression to the old http_code/000 logic. helm-unittest 248/248; lint clean. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(egress-proxy): probe reports DNS failure distinctly, not as "TCP connect succeeded" [#104] (#256) Bugbot (PR #254): the non-enforcement WARNING ("...reached :443 ... the TCP connect succeeded") was emitted for every non-{7,28} curl exit code, including exit 6 (DNS resolution failure) where no TCP connection was ever attempted — so the failure output could contradict what actually happened and mislead troubleshooting. Split the verdict into a case: 7|28 => TCP never established => egress blocked (pass, exit 0) 6 => INCONCLUSIVE: could not resolve host, no TCP attempted => fail (exit 1) with an accurate message (never claim "enforced" on a DNS failure) else (0, or TLS-/HTTP-layer errors 35/52/56/60 that all require a completed TCP connect) => host reached => NOT enforced (exit 1) Default probe host is an IP, so exit 6 only arises for a hostname probe on a cluster with broken DNS. helm-unittest 248/248; lint clean. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
… backend-reachability test, installer hardening) (#273) * Merge pull request #260 from tracebloc/fix/job-resource-defaults-745 test(jobs-manager): lock spawned-job RESOURCE_REQUESTS/LIMITS default at 8Gi * test(e2e-proxy): exercise application-pod egress through the proxy (Charité setup) (#264) * test(e2e-proxy): exercise application-pod egress through the proxy The squid harness proved NODE egress (image pulls) but stopped before any application pod — so it never caught client-runtime#119, where the spawned ingestion Job carried no proxy env and dialled the backend directly. Add a section that runs a pod WITH the ingestion-style proxy env (must traverse the squid to reach the backend) and a pod WITHOUT it (must bypass it / go direct), asserting both against the squid access log. Models the Charité proxy-only setup at the application layer; pairs with the behavioural unit tests on client-runtime#119. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(e2e-proxy): rework app-pod egress to an in-cluster squid Service Running the first version on a real k3d cluster surfaced that a POD cannot resolve host.k3d.internal (it is a node-level alias for image pulls, not pod DNS), so the proxied probe failed with `curl (5) Could not resolve proxy`. Rework: stand up an in-cluster squid Deployment+Service the test pods reach by Service DNS (also a closer model of a real corporate proxy reachable by name), with a readiness probe gating rollout on squid actually listening (fixes the probe-before-bind race seen in the first attempt). A pod WITH the ingestion proxy env must reach the backend through the squid; a pod WITHOUT it must bypass it. Auth survival stays covered by the host-squid sections (1-3). bash -n + shellcheck + embedded-YAML parse all clean; Service-DNS resolution verified locally. Full proxied-curl run is exercised by the e2e-proxy CI job. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(e2e-proxy): make app-pod egress assertion deterministic (single pod, curl -v) §4 now uses ONE pod carrying the ingestion-style proxy env that makes two calls to the same backend: WITH the env it must tunnel via the in-cluster squid (a CONNECT tunnel); with the env unset it must dial direct. Proof is taken client-side from `curl -v` (the CONNECT-tunnel lines), not by reading squid's access.log — that file is buffered by the log daemon and came back empty when read right after the probe, producing false failures. Also set BOTH proxy-env cases: curl honours the lower-case `https_proxy` for HTTPS and the upper-case alone is not reliably picked up, so the probe must emit both — exactly as the real ingestion env does. A single pod with a single log also removes the multi-pod scheduling / log-flush races that made the earlier two-pod form flaky. Validated end-to-end on k3d: A (proxy env) -> "Establish HTTP proxy tunnel to api.tracebloc.io:443" + "CONNECT tunnel established, response 200" + 200 OK B (env unset) -> direct connect to the backend IP, no proxy tunnel, 200 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * ci(security): add public-repo PII gate caller (#263) Blocks PRs that leak customer/partner names or secrets in title/body/commits. Calls the reusable gate in tracebloc/.github. Inactive until the org PII_DENYLIST secret is set (warns, doesn't block, until then). * ci: add concurrency cancellation + job timeouts to chart CI (#265) helm-ci.yaml and installer-tests.yaml are the repo's most expensive workflows (a real k3d cluster, a 9-distro docker-in-docker matrix, Windows Pester) but had no concurrency control, so a PR re-push left stale runs burning to completion. Add a per-ref concurrency group that cancels superseded PR runs only (push/schedule runs are never cancelled), matching the pattern already used in client-runtime's tests.yml. Add timeout-minutes to every job so a hung k3d/squid/distro step can't run to the 6h default. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): fail fast when HOST_DATA_DIR is on a network filesystem (#261) Detect NFS/CIFS/SMB for HOST_DATA_DIR in preflight (bash + PowerShell) and fail fast with an actionable message instead of a cryptic MySQL CrashLoopBackOff ~20 min into install: MySQL/InnoDB corrupts on network storage and the chart root chown init-container is blocked by NFS root_squash. - preflight.sh: _pf_fstype reader (findmnt, then GNU stat, then df+mount; portable incl. macOS) + _pf_storage_type wired into run_preflight. Allowlists network fstypes so local FSes including overlay/tmpfs (CI) pass. - install-k8s.ps1: Get-PfFsType (UNC / network drive) + Test-Preflight check. - TRACEBLOC_ALLOW_NETWORK_FS=1 overrides (mirrors TRACEBLOC_ALLOW_ARM64). - Tests: 10 bats cases + Pester cases (network -> fail, override, undetermined, Windows-only Get-PfFsType reader). Part 1 of 3 for tracebloc/backend#743. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(installer,chart): place datasets on a network mount while MySQL stays local (#262) * feat(installer): fail fast when HOST_DATA_DIR is on a network filesystem Detect NFS/CIFS/SMB for HOST_DATA_DIR in preflight (bash + PowerShell) and fail fast with an actionable message instead of a cryptic MySQL CrashLoopBackOff ~20 min into install: MySQL/InnoDB corrupts on network storage and the chart root chown init-container is blocked by NFS root_squash. - preflight.sh: _pf_fstype reader (findmnt, then GNU stat, then df+mount; portable incl. macOS) + _pf_storage_type wired into run_preflight. Allowlists network fstypes so local FSes including overlay/tmpfs (CI) pass. - install-k8s.ps1: Get-PfFsType (UNC / network drive) + Test-Preflight check. - TRACEBLOC_ALLOW_NETWORK_FS=1 overrides (mirrors TRACEBLOC_ALLOW_ARM64). - Tests: 10 bats cases + Pester cases (network -> fail, override, undetermined, Windows-only Get-PfFsType reader). Part 1 of 3 for tracebloc/backend#743. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(installer,chart): place datasets on a network mount while MySQL stays local Storage split for VMs whose real storage is an NFS/CIFS mount (backend#743): the database must stay on local disk (InnoDB over NFS is unsafe) but the large dataset volume can live on the network mount. Chart: - Parameterize the dataset PV hostPath base via hostPath.datasetPath (helper tracebloc.clientDataHostPath). Default /tracebloc keeps it byte-identical; mysql + logs PV paths are unchanged. values.yaml + schema + nil-guard for --reuse-values upgrades. Installer (bash + PowerShell): - New HOST_DATASET_DIR: validated (must exist + be writable; MAY live outside $HOME unlike HOST_DATA_DIR; system paths barred), bind-mounted into k3d at a distinct /tracebloc-data path; the dataset dir is created there while mysql + logs stay local. When set, the generated values set hostPath.datasetPath=/tracebloc-data and (Linux) pass HOST_UID/HOST_GID env to jobs-manager so spawned ingestion pods write the host-owned NFS export as the owning uid. Preflight notes the dataset dir is exempt from the network-FS block. Tests: new shared_images_pvc_test.yaml + mysql/logs split-only guards (helm-unittest, 259 pass); HOST_DATASET_DIR validation, second-mount, dir-split and values-generation cases (bats). Docs: INSTALL.md checklist + SECURITY.md 5.4. Part 2/3 of backend#743. The end-to-end NFS write path also needs the client-runtime ingestor-uid change (separate PR) so jobs-manager reads HOST_UID. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(#262): fail fast when an existing cluster lacks the dataset bind mount (Bugbot) The HOST_DATASET_DIR -> /tracebloc-data bind mount is baked into the k3d nodes at create time (_create_new_cluster / the PS1 equivalent). k3d cannot add a mount to a RUNNING cluster, but install-client-helm.sh still wrote `datasetPath: /tracebloc-data` into the generated values whenever HOST_DATASET_DIR was merely set — so an existing-cluster re-run pointed the chart's dataset PV at ephemeral in-node storage, silently putting datasets on disposable storage instead of the network export (lost on a restart). Add _check_existing_cluster_dataset_mount (cluster.sh) + the PowerShell equivalent, mirroring the existing _check_existing_cluster_proxy/bind drift checks: on an existing cluster with HOST_DATASET_DIR set, inspect the server node for the /tracebloc-data mount and FAIL FAST with the recreate remedy if it is absent — rather than installing a quietly misrouted dataset volume. Fail-fast (not warn) because this is silent data loss, consistent with the network-FS fail-fast guard. Values generation needs no change: the install now stops in Step 2, before helm runs. +4 bats (cluster.bats): unset -> no-op, mount present -> pass, mount ABSENT -> fail fast, inspect fails -> no-op. bash + shellcheck clean; pwsh parses the .ps1; full cluster suite 27/27. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(installer): non-interactive credentials + softer connect copy (#834) (#266) RFC-0001 Phase 0 (no backend dependency): - Accept TRACEBLOC_CLIENT_ID / TRACEBLOC_CLIENT_PASSWORD so CI / automation / golden images can provision without typing the secret inline. Verified the same way as the prompt (verify_credentials); a bad credential fails the install (no re-prompt in non-interactive mode). The interactive prompt path is unchanged — just wrapped in the else branch. - Soften the "to connect this machine you NEED a tracebloc client / create one" copy to "already have one? enter it (or set the env vars) / need one? create it" — so we stop framing client-creation as a mandatory pre-step (browser sign-in lands in Phase 1). Tests: two new bats cases (env path -> non-interactive write + helm with no prompt; rejected env creds -> error, no helm). The interactive flow tests (re-prompt / inactive / unverified / defaults / max-attempts / one-client guard) still pass unchanged. NOTE: bats #16 (_extract_yaml_value single-quote '' un-escape) fails locally on macOS bash — pre-existing and untouched by this PR (the diff doesn't go near that function); flagging for confirmation against CI. Part of backend#830. Closes #834. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Asad Iqbal <asad.dsoft@gmail.com> * test(e2e-proxy): deflake §4 app-egress — hermetic target + non-silent diagnostics (#269) §4 ("APPLICATION-pod egress through a proxy", client-runtime#119) was a flaky required check ("E2E auth-proxy (squid)") that intermittently red-X'd develop (~1 in 4; e.g. run 27765964135) and randomly blocked unrelated PRs. Two causes: 1. Silent failure. Under `set -euo pipefail` the diagnostic `grep | sed` lines ran before the real assertion; an empty section made grep exit 1 → pipefail → set -e killed the script with NO output (CI showed only "pod/egress-app created" then "exit code 1"). Append `|| true` so the diagnostics are non-fatal and the assertion fires with its reason. Same footgun fixed in §3. 2. External-network dependency (the real flake). §4 curled the real https://api.tracebloc.io/ through the in-cluster squid, depending on the runner's internet to a production host at test time. Make it hermetic: target a reserved-TLD stand-in host (backend.tracebloc-e2e.test) aliased via hostAliases on both the squid and app pods to the cluster's own kube-apiserver ClusterIP — a guaranteed in-cluster HTTPS:443 listener. The CONNECT tunnel now terminates in-cluster with zero external I/O, preserving the #119 intent (WITH proxy env → CONNECT tunnel via squid; env unset → direct dial). Validated: 3/3 deterministic local passes; both calls hit 10.43.0.1 in-cluster (no api.tracebloc.io reachout). bash -n + shellcheck --severity=error clean. Closes #268 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(cli#90): in-cluster backend-reachability helm test (WS3) (#270) Adds `egress-reachability-check`, a `helm test` Job that verifies a normal (non-training) pod in the namespace can reach the tracebloc backend API — the egress dependency that gates everything (the cluster authenticates to the backend to obtain its Service Bus credentials, so no backend egress => silent Pending). The required-egress complement to egress-enforcement-check (which verifies the opposite: that training pods are locked out). The probe is deliberately NOT training-labelled (so the lockdown netpol never selects it — it keeps the jobs-manager/requests-proxy egress class) and honours tracebloc.proxyEnv, so it tests the real path. The verdict keys on curl's exit code (TCP reachability), not HTTP status. Run via `helm test <release>`; gated by egressReachabilityCheck.enabled (default true; disable on truly air-gapped clusters). As a test hook it never runs during install/upgrade. Service Bus is intentionally not probed here: its host is fetched post-auth from the backend (static nowhere in the chart) and its egress is brokered by the requests-proxy, whose readiness `tracebloc cluster doctor` already checks. helm-unittest: 6 tests (render / disable / test-hook annotation / not-training- labelled / CLIENT_ENV-driven host / proxy-inherited). Full chart suite green. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(chart): bump client 1.7.1 → 1.8.0 (version + appVersion) (#272) Release v1.8.0 — promotes the 9 commits merged to develop since v1.7.1. Minor bump: #262 (datasets on a network mount while MySQL stays local) is a real PVC-placement change at install time, not inert. Keeps version/appVersion in lockstep so the app.kubernetes.io/version label matches the release. Refs #271 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(egress-reachability-check): fail closed on TLS/transport errors, not just connect failures (#274) The backend-reachability helm test (#270) keyed success on a denylist of curl exit codes (5/6/7/28) and passed on everything else. But with no --fail, curl exits 0 for any HTTP response, so the only non-zero codes are transport/TLS failures — yet a TLS handshake/cert error (e.g. 35/51/60, typically a proxy intercepting TLS with a CA the cluster doesn't trust) printed "backend reachable" and exited 0: a false pass that defeats the check's purpose. Pass ONLY on curl exit 0 (a full TCP+TLS+HTTP round trip proves the backend is reachable and usable); fail closed on everything else, with a dedicated TLS/cert bucket and a generic catch-all. Companion helm-unittest asserts nothing on the script body; full suite 265/265. Follow-up to #270; flagged by Cursor Bugbot on the v1.8.0 sync PR (#273). Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: lukasWuttke <54042461+LukasWodka@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…k standard-mode CNI fix) (#278) * fix(egress-enforcement-check): retry probe to tolerate standard-mode CNI reconciliation window (#276) * fix(egress-enforcement-check): retry probe to tolerate standard-mode CNI reconciliation window The SECURITY §8.2 egress-lockdown enforcement helm test false-failed on clusters whose CNI enforces egress NetworkPolicy in "standard" mode — notably AWS VPC CNI with NETWORK_POLICY_ENFORCING_MODE=standard, which both the dev and prod fleets use. Standard mode allows a brand-new pod all traffic until its per-pod policy reconciles (a few-second startup window); the probe curled the canary host instantly on startup, won that race, connected, and reported "EGRESS LOCKDOWN NOT ENFORCED" — even though the lockdown is enforced for real training pods, which start far slower than the window. Retry the probe until egress is observed blocked, bounded by a new networkPolicy.training.enforcementProbeTimeoutSeconds (default 60s): PASS as soon as the connect is blocked, FAIL only if the host stays reachable for the whole timeout. Correct on standard mode, strict mode, and genuinely non-enforcing clusters alike. Also refresh the stale values.yaml CNI guidance (AWS VPC CNI does enforce with enableNetworkPolicy=true) and extend helm-unittest (full suite 267 pass). Verified on dev (EKS, tb-client-dev-templates): a training-labelled pod that sleeps 30s then probes is blocked on 1.1.1.1:443 and example.com:443, and reaches the backend via the squid gateway. Refs #275, tracebloc/client-runtime#104, tracebloc/client-runtime#102. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(egress-enforcement-check): NOT-ENFORCED verdict outranks a late DNS failure Bugbot (PR #276): the post-loop inconclusive branch keyed off only the last curl exit code. If an earlier attempt observed the host reachable (rc=0) but a later attempt failed DNS (rc=6), the hook reported "inconclusive / no TCP attempted" instead of "NOT ENFORCED" — a misdiagnosis that could send an operator chasing DNS when the CNI simply isn't enforcing egress. Latch a `reached` flag whenever a probe establishes a TCP connection (any curl exit other than 6) and branch on it after the loop: a connect seen on ANY attempt yields the definitive NOT-ENFORCED verdict; INCONCLUSIVE is reserved for the case where no TCP connection ever succeeded. helm-unittest updated. Refs #275. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * chore(chart): bump client 1.8.0 → 1.8.1 (version + appVersion) (#277) Readies the 1.8.1 release, which ships the egress-enforcement-check standard-mode CNI fix (#276, closes #275): the §8.2 helm-test gate now retries through the per-pod NetworkPolicy reconciliation window instead of racing it, so it no longer false-fails on AWS VPC CNI standard mode (the dev and prod fleets both use it). Validated end-to-end on dev — the patched `helm test` goes green where the previous instant-probe test failed. Refs #275. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…sion reorder (#284) Promotes #279 (installer reorder: sign-in + provision before Helm), #281 (R8 verified bootstrap + sign-installer-manifest release job), #285 (stale-bootstrap CLI install fix), #286 (jq-free one-client guard). Brings the R8 install.sh + the sign-installer-manifest job to main ahead of the signed installer release. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…evelop (#290) fix(installer): two R8 curl|bash install P0s — stamper guard-clobber + provision --name (→ v1.8.2)
chore(release): promote develop → main — restore verify canary + adopt-reconcile (v1.8.4)
chore(release): promote develop → main — R8 Windows installer + trust-root hardening (v1.8.5)
main: Fix installer issues and enhance first-run user experience
chore(chart): bump client 1.8.5 → 1.9.0 (version + appVersion)
chore(chart): roll ingestor tag 0.5 → 0.6, bump chart 1.9.0 → 1.9.1
… + curl|bash survival) (#327) * Merge pull request #325 from tracebloc/fix/cred-leak-curl-argv fix(installer): stop leaking the client password on curl's argv (CWE-214) * fix(installer): survive curl|bash — retry name prompt (customer-reported), read creds from terminal, guard pkg-index refresh (#326) * fix(installer): don't abort Linux install when the package-index refresh flakes install_system_deps ran `spin_cmd "Updating package index…" $PM_UPDATE` unguarded — under set -e a transient mirror/network failure there aborted the whole install. Yet the per-package installs right below are already guarded (|| log), so a flaky refresh was MORE fatal than a failed install, which is backwards: a stale index usually still installs from cache. Guard it with || warn so we continue to the (guarded) installs, which surface a genuinely missing package with an actionable message. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): read credential prompts from the terminal so curl|bash doesn't abort The Step-5 credential prompt read from stdin. Under `curl … | bash` stdin is the piped script, not the terminal, so each `read` hit EOF and (under set -e) aborted the installer with an opaque failure the moment it reached the prompt — the dual-mode env-var path (TRACEBLOC_CLIENT_ID/PASSWORD) was the only way through, but nothing told the user that. Read prompts from TB_TTY (the controlling terminal, /dev/tty) instead, the same mechanism provision.sh already uses. When no terminal is available and no env creds were supplied, fail with an actionable message pointing at TRACEBLOC_CLIENT_ID/PASSWORD rather than the set -e abort. TB_TTY is overridable so the bats suite can feed canned input on stdin. Regenerated scripts/manifest.sha256 (R8 supply-chain: any scripts/ change must re-pin, alongside the setup-linux.sh guard in the previous commit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): retry the client-name prompt so tty type-ahead can't abort provisioning Customer-reported 2026-07-09 (external onboarding): the installer signed in, printed the name + location prompts, captured NEITHER, and died with 'A name for this client is required to provision it.' provision_client read the name with `IFS= read -r client_name </dev/tty || true` — a single shot whose status was swallowed by `|| true`, with no retry and no fallback (location silently defaults to the detected zone, which is why only the name failed hard). Any empty/failed read on the name → the fatal error. The most likely trigger is tty type-ahead: during the ~minute browser-approval wait the CLI reads nothing, so a stray newline queued in the terminal is consumed by the read as an empty name. (An adversarial pass ruled out the background-process-group and login-drains-the-tty theories; the reader is foreground and login never touches the tty — the empty read is an environmental dead/queued-input condition.) Fix: read the name in a bounded retry loop that RE-PROMPTS on an empty line (so a queued blank is skipped, not accepted) and BREAKS on a failed read (rc!=0 = EOF / no live input, which re-prompting can't fix) so the actionable 'set TRACEBLOC_CLIENT_NAME' error still fires. Reads route through TB_TTY (defaults to /dev/tty; overridable so the bats suite can feed stdin), matching the install-client-helm.sh credential reads in this branch; prompt WRITES stay on /dev/tty but are guarded so a test without a real terminal doesn't abort. The location reads adopt TB_TTY too (their empty->fallback behavior is unchanged). 2 new provision.bats tests: type-ahead blanks are re-prompted then the real name is captured; a dead-input tty (EOF) fails fast with the guidance. Regenerated scripts/manifest.sha256 (R8). NOTE: this is Failure 1 of the report; the existing install-client-helm.sh fix on this branch does NOT cover these provision.sh reads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): guard credential reads against a dead-input tty (EOF), not just a missing one Review feedback on #326 (Asad + Bugbot): the credential reads route through $TB_TTY but had no EOF guard, unlike the provision.sh name read (which breaks on rc!=0). _tty_available only checks `-r`, so on a readable-but-dead-input tty (non-PTY ssh, an IDE terminal, a drained/queued tty — the same class this PR documents for provision.sh) it returns true, the actionable no-creds error is skipped, and the first `read <"$TB_TTY"` hits EOF and aborts under set -e — the exact opaque failure this PR set out to remove, left in place for creds. Factor the actionable env-var guidance into _no_interactive_creds_die and call it from BOTH the `! _tty_available` check AND a per-read `|| _no_interactive_creds_die` guard on all five prompts (the Use-previous read + the ID/password reads). A dead-input tty now fails fast with the same guidance as no-tty instead of aborting mid-read. Happy path (input present) is unchanged — the guard only fires on EOF; all existing cred tests (re-prompt, reuse-defaults, max-attempts) consume their fed input exactly and still pass. +1 regression test (readable /dev/stdin backed by /dev/null → EOF → actionable error, no helm). Regenerated scripts/manifest.sha256 (R8). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * chore(chart): bump client 1.9.1 → 1.9.2 (version + appVersion) (#329) Publishes everything unreleased since v1.9.0: - #323 ingestor default tag → 0.6 (landed as 1.9.1, never released) - #325 stop leaking client password on curl's argv (CWE-214) - #326 survive curl|bash: retry name prompt, read creds from terminal, guard pkg-index refresh (customer-reported) Refs #328 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: lukasWuttke <54042461+LukasWodka@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sync develop → main for v1.9.3 chart release (ingestor 0.7)
…digest pinning (#345) * test(provision): pin the #303 pre-flight's grep contract with hidden `client list` (Refs #141) (#335) The installer's #303 one-client-per-machine pre-flight (`scripts/lib/provision.sh` `_account_owns_namespace`) shells out to `tracebloc client list --plain` and greps the output for `namespace=<ns>([[:space:]]|$)` to refuse a cross-account re-provision. In the cli repo `client list` is now a HIDDEN cobra command — still callable, but nothing here pinned that the pre-flight keeps classifying the CLI's exact --plain output correctly, nor that it still passes --plain. If either drifts the grep silently fails and #303 stops firing. Add two focused unit tests of `_account_owns_namespace` (the consumer half of the cli#141 contract; the producer half is pinned in the cli repo at internal/cli/client_list_contract_test.go). The fixture mirrors, field-for-field, what cli's runClientList prints under --plain: - owned namespace → rc 0; absent → rc 1; a STRICT PREFIX of a real namespace → rc 1 (pins the ([[:space:]]|$) anchor: an account must not "own" acme-prod-0 just because it owns acme-prod-01); - the pre-flight actually invokes the hidden list WITH --plain; - an unreadable list is rc 2 (fall through to create), distinct from rc 1 (refuse) — the distinction #303 branches on. Assertions use `[ -eq ]` + `grep` (real exit codes, robust on bash 3.2); Linux CI (standard-checks.yml → `bats scripts/tests/*.bats` on ubuntu) is the authority. Mutation-verified: dropping the grep anchor or --plain from provision.sh fails these tests. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(proxy): NO_PROXY covers the cloud metadata IP 169.254.169.254 (#337) The auto-augmented NO_PROXY lists (chart tracebloc.proxyEnv helper, bash _augment_no_proxy defaults, Windows Get-EffectiveNoProxy defaults) omitted 169.254.169.254 — behind a corporate proxy, cloud metadata lookups would be routed through the external proxy (broken IMDS access and an unnecessary place for instance credentials to transit). Add the IP to all three lists in lockstep, pin it in the bats + Pester + helm-unittest expectations, regenerate manifest.sha256 for the two touched installer scripts, bump chart 1.9.3 -> 1.9.4. Ref tracebloc/backend#803 (item J), tracebloc/client-runtime#120 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * feat(installer): stop-and-check gate — recognize an existing install (#339) * feat(installer): stop-and-check gate — hand a healthy re-run to the home screen Re-running the installer on an already-set-up machine no longer drags the user through full provisioning. A new read-only gate (scripts/lib/assess.sh) runs after the banner and classifies the machine: • healthy — cluster running AND a tracebloc release present AND jobs-manager Ready AND the CLI present → print "Already set up on this machine", hand off to `tracebloc` (the home screen), exit 0. • degraded — cluster stopped / workload not Ready / CLI missing / any partial state → print an honest one-liner, fall through to the normal flow to reconcile. • fresh — no cluster, or a cluster with no release → the normal flow. assess is STRICTLY non-mutating and bounded: read-only `k3d cluster list`, `helm list`/`get values` (reuses detect_installed_client), and a bounded `kubectl get` (--request-timeout). On ANY uncertainty it degrades toward the normal flow — never a false "healthy" that would skip a needed install. The hand-off uses `exit 0` (not exec) so the EXIT-trap cleanup still runs; if the CLI is somehow unresolvable it falls back to a status line and still exits 0. --force / --reinstall (or TRACEBLOC_FORCE_REINSTALL=1) bypasses the gate. A healthy machine still short-circuits under curl|bash (output only, no input). Per-layer surgical repair of a degraded machine is a deliberate fast-follow so this PR stays off provision.sh (clear of the active #838 work). Wires the gate into main() (after print_banner, before the roadmap), adds scripts/lib/assess.sh to the bootstrap FILES + gen-manifest, regenerates scripts/manifest.sha256, and adds scripts/tests/assess.bats. New copy says "secure environment", never "client". PR 2 of 2 (PR 1 = the cli home screen, cli#244). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): assess "healthy" = ALL client workloads Ready + guard awk branch Code-review should-fix items on the stop-and-check gate: 1. _assess_cluster_servers_running: add `|| line=""` to the awk-branch assignment, mirroring the jq branch. awk's `exit` closes the pipe, so under `set -o pipefail` a SIGPIPE from k3d (141) — or any k3d failure — would otherwise propagate non-zero out of the assignment and abort the installer. 2. "healthy" must match the installer's OWN definition of ready. The probe now requires ALL the workloads wait_for_client_ready checks — mysql-client, ${ns}-jobs-manager, ${ns}-requests-proxy — not jobs-manager alone. Previously a machine with jobs-manager up but requests-proxy (training egress) or mysql-client down was classified healthy and short-circuited without reconciling — the false-positive we designed against. Single source of truth: the deployment set is extracted into _client_workload_deployments (common.sh); both wait_for_client_ready (summary.sh) and the assess gate consume it, so they can't drift. Renamed _assess_jobs_manager_ready -> _assess_workload_ready (reason stays `workload-not-ready`); any one workload not-Ready/absent -> degraded. Tests: _assess_workload_ready now covers all-three-Ready plus each workload individually down/absent; classify covers "one down -> degraded" via the real probe and "all three Ready + CLI -> healthy". Mutation-checked: shrinking the shared list to jobs-manager-only fails the mysql-client/requests-proxy/one-down tests. Manifest regenerated (assess.sh, common.sh, summary.sh). provision.sh untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): run the helm client install behind a spinner (no more silent ~10-15s) (#333) * fix(installer): run the helm client install behind a spinner (no more silent ~10-15s) Step 4/5 ran `helm upgrade --install` (and the in-place reconcile) blocking, with all output redirected to the log, so the terminal sat frozen ~10-15s (render + apply + image pull) with no feedback. Wrap both in the existing spin_cmd helper: an animated spinner + message, output still streamed to $LOG_FILE, and on failure the log tail to stderr + the same error-exit. Honours RFC-0002 §2 "progress on every wait" (the principle already applied to the CLI's post-submit waits). - bash -n + shellcheck clean; install-client-helm.bats green (33 tests). - Failure path preserved (error-exit + full output in $LOG_FILE). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(scripts): refresh integrity manifest for spinner change gen-manifest.sh after the install-client-helm.sh spinner edit; static-analysis CI gate requires the sha256 manifest to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * chore(chart): prod-only ingestor digest pin (#1028 item 1) (#334) * chore(chart): prod-only ingestor digest pin via values-prod overlay (#1028) Prod deploys the ingestor image by immutable digest; dev/staging keep floating :0.7 (imagePullPolicy=Always). Base values.yaml keeps digest empty; the new client/values-prod.yaml sets only images.ingestor.digest and is layered at prod-release time (-f values.yaml -f values-prod.yaml). Digest verified live against ghcr.io (multi-arch index, amd64+arm64). Adds resolve-ingestor-digest.sh to re-resolve/verify the digest at each cut instead of hand-typing it. Refs tracebloc/backend#1028. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(digest): --write refuses single-arch pin + verifies sed landed Multi-arch guard now inspects the resolved repo@digest and hard-fails under --write instead of only warning (a single-arch pin breaks arm64 and fails helm-ci ingestor-multiarch). --write also greps the overlay for the digest after sed and errors on a silent no-op instead of falsely reporting success. Bugbot findings on client#334. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(digest): read chart tag without yq; fail loudly, never hardcode 0.7 The no-arg TAG default silently fell back to a hardcoded 0.7 when yq was absent, so a prod cut after the chart tag moved could --write a stale/wrong digest while appearing to follow the chart. Reads images.ingestor.tag via yq when present, else a portable awk parse scoped to the images:->ingestor: block; if neither works it errors and exits non-zero. Bugbot follow-up on client#334. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(digest): tolerate pipefail in the multi-arch platforms capture Under set -o pipefail, a failed inspect or a grep -v that filters every line exits non-zero and aborted the whole script even after the digest resolved. Add || true; an empty platforms then trips multiarch=0 so the guard still fires (ERROR under --write, WARNING otherwise). Bugbot round-3 on client#334. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * docs(security): update §8.3 — backend tokens now bounded + revocable (#302) (#342) §8.3 claimed "Backend tokens never expire"; that finding is now mitigated for interactive DS web sessions via a bounded, revocable 30-day ClientAccessToken (backend#933 + frontend-app#575, shipped through backend#590). Document the mitigation, the intentional carve-out for edge devices / bots (which keep the legacy long-lived DRF Token as non-interactive service credentials, by design), and the residual JS-readable-storage risk tracked as the SEC-06 follow-up in tracebloc/backend. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): Windows one-client guard reads clientId via ConvertFrom-Json (#200) (#340) The one-client-per-machine guard regex-scraped `helm get values` YAML and only matched a DOUBLE-QUOTED clientId. Helm re-serializes values on `get`, so typical clientIds come back unquoted -- the guard silently matched nothing and a re-install could re-point the machine to a different Client ID, defeating the protection from #192 (Bugbot finding on #199). Read the values as JSON instead (`helm get values <rel> -n <ns> -o json | ConvertFrom-Json` -> `.clientId`), which sidesteps YAML quoting entirely and matches the adjacent comment that already claimed ConvertFrom-Json. A release with no user values (literal `null`), a missing clientId key, or unparsable output from one release is skipped without aborting the scan of the remaining releases. Tests: existing guard mocks now serve real-helm-shaped output (JSON for `-o json`, unquoted YAML otherwise), plus new cases for unquoted / single- quoted / double-quoted YAML views, a null-values release mid-scan, and values without a clientId key. Pester (Linux container, lts-7.4): 95 passed, 0 failed, 6 skipped ($IsWindows skips). scripts/manifest.sha256 regenerated (install-k8s.ps1 line only) -- the file is on the signed-manifest surface; gen-manifest.sh --check passes. Closes #200 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * docs(supply-chain): document the Windows bootstrap (install.ps1) (#336) * docs(supply-chain): document the Windows bootstrap (install.ps1) SUPPLY_CHAIN.md scoped itself to install.sh with zero Windows mentions, while the R8 Windows leg (PR #299, released v1.8.5) has shipped the same guarantee for install.ps1. Add the Windows section, the install.ps1 release asset to the asset table, and a PowerShell verify-by-hand note. Closes tracebloc/backend#957 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(supply-chain): correct the raw-<TAG> Windows bootstrap (fails closed w/o $env:REF) Arturo's review (client#336): the §7 parenthetical told customers they could bootstrap install.ps1 from a pinned <TAG> raw URL, but the committed tag tree still ships the __TRACEBLOC_RELEASE_REF__ placeholder (only the release asset is stamped), so that path refuses to run unless $env:REF is set. Documented the $env:REF requirement + why, instead of implying the raw <TAG> URL works as-is — aligned with §1's $env:REF description and the bash side (which uses raw-<TAG> only for hand-verifying a sub-script against the manifest, not for bootstrap). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * feat(installer): first-time install run-through (1. Downloading / 2. Installing a–f) + healthy re-run bailout (#341) * feat(installer): first-run banner/roadmap + step_header + count_bar helpers - print_banner: new title 'Setting up tracebloc on your machine · <version>' (tracebloc bold-cyan) + rule; TB_VERSION from TRACEBLOC_INSTALL_REF; skips when the bootstrap already drew it (TRACEBLOC_BANNER_SHOWN). - print_roadmap: the '2. Installing' a–f plan. - step_header: bold gerund running headers for steps a–f. - count_bar: honest N-of-M render helper for multi-image pulls. - preflight_sudo: step-b password intro copy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): bootstrap early-bailout + "1. Downloading" run-through - Early bailout: if the tracebloc CLI is present and `tracebloc doctor` reports healthy (bounded, exit-code gated), print the healthy line and exec the home screen — skipping the download entirely. Skipped on --force/--reinstall, TRACEBLOC_FORCE_REINSTALL, the dev/unverified path, or an explicit REF/BRANCH. - Draw the first-run banner here and export TRACEBLOC_BANNER_SHOWN (so install-k8s.sh does not draw a second) + TRACEBLOC_INSTALL_REF. - "1. Downloading" copy: Installer downloaded — N files / Verifying it's authentic (cosign)… / Signature verified / All N files intact — nothing was altered. Local colour palette (common.sh not yet sourced). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): step a — collapsed hardware summary + connectivity spinner - run_preflight renders the run-through's step-a view: one hardware line ('arch · N CPU cores · N GB memory · N GB free disk'), a connectivity spinner + combined 'Connected: …' line, and a 'Local storage (~/.tracebloc)' line. - PF_QUIET_SUCCESS suppresses the per-check ✔ lines only inside run_preflight (folded into the summary); called directly (bats), the checks still print their ✔/info lines, so the unit contracts hold. Warnings + hard-fails always print. - Connectivity probes stay in the foreground (PF_HARD_FAIL propagation) with a no-sleep per-host spinner frame. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): step b — Docker %-bar copy + 'ready' success lines - Real %-by-bytes bar for the Docker Desktop .dmg (single-file curl via download_with_progress); fresh-Mac intro copy on the label. - 'Docker ready' and 'System tools ready (k3d, helm, kubectl)' to match the run-through. Linux (Docker Engine) copy left untouched — a deliberate follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): step c — spinner around the (silent) k3d create - _create_new_cluster wraps the 1-2 min k3d create in a spinner ('Creating your secure environment…'), the real fix for the long silent gap; spin() waits for the backgrounded create so exit-code capture + proxy-config cleanup are intact. - Runtime intro copy; terminology 'compute environment' → 'secure environment'. - _wait_for_api owns the single 'Secure environment ready' (API confirmed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): step d — device-flow sign-in copy + "Registered as" - Reworked sign-in copy: "open the link on this or any device and enter the code" (print-only; the CLI prints the URL/code/wait — no auto-open claim). - Success line → Registered as "<slug>" (the minted namespace = dashboard name). - CLI install call removed (now step b); keeps the has-tracebloc FATAL guard. - KEPT the interim name/location prompt (deployed CLI still hard-requires --name without a TTY, backend#992) + comment: remove when cli#137 ships. - No internal step header (main() prints "d) Registering this machine"). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): step b — "tracebloc CLI ready" on the fully-clean path Fully-clean verdict (usable now + in new terminals) → "tracebloc CLI ready … verified on your PATH", matching the run-through's Docker/System-tools/CLI "ready" pattern. Edge-case lines stay "installed" (installed but not yet usable here). Test assertion updated to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): step e — services count-bar + "tracebloc installed" - Services intro copy (training runner / data manager / live monitor / local database; runs on your machine, data never leaves). - _download_services_progress: honest N-of-M count bar as service images pull (imageID-populated count), bounded + non-fatal; guarded by TB_NO_SERVICE_PROGRESS (set in the bats setup so the mocked-kubectl poll cannot hang). Never a fabricated aggregate %. - Success line "Connected to tracebloc" → "tracebloc installed" (step f/summary owns "Connected"); dropped the internal step 4/5 headers (main owns a–f). - Tests: retarget the 3 assertions + guard the poller. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): step f — rich connected summary matching the run-through - connected summary: "✔ Connected to tracebloc"; Environment/Version/Mode block; "live 🟢" + dashboard; NON-dim "What's next" with 3 numbered steps (tb data ingest / my-use-cases / invite collaborators); prominent "Run tracebloc to get started."; dim footer (Logs · Data) with the reboot tip as the LAST dim line. Dropped the green ━━━ border. - Terminology: "secure environment"; trust claim "never leaves this machine". - _reboot_note stays OS-guarded (Linux: restarts automatically; macOS: open Docker Desktop) — Linux not regressed. - wait_for_client_ready intro reframed as step-f "Connecting…". - Tests updated to the new copy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(installer): main() reorg to the six-step run-through (a–f) + gate slot - main() now: banner → [client#339 gate SLOT] → roadmap → a) Check your machine → b) Install what tracebloc needs → c) Create your secure environment → d) Register this machine → e) Install tracebloc → f) Connect to the tracebloc network, each with a step_header + trailing blank-line pair. - Step b now owns prerequisites AND the tracebloc CLI (moved out of provisioning; step d needs it to sign in). - Gate slot: guarded NO-OP call to assess_existing_install, clearly commented as client#339's — logic NOT implemented here; positioned after banner/before roadmap so it reconciles cleanly with that branch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(installer): regenerate manifest.sha256 for the first-run UX changes Content hashes for the 9 edited sub-scripts (common/preflight/setup-macos/ cluster/provision/install-cli/install-client-helm/summary/install-k8s). No files added or renamed, so install.sh's FILES array is unchanged; gen-manifest.sh --check passes (both bootstrap-sync checks green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(installer): cover the first-run UX logic (Linux CI is the authority) - common.bats: count_bar (render/clamp/bad-input/divide-by-zero), step_header, print_roadmap (a–f plan), print_banner (version + bootstrap-suppression). - preflight.bats: _pf_hw_summary_line, connectivity combined 'Connected:' line, run_preflight collapsed step-a view (per-check ✔ lines folded away). - install-client-helm.bats: _download_services_progress guards (TB_NO_SERVICE_ PROGRESS / no-kubectl / empty-ns) so the poller can never hang the suite. - install-bootstrap.bats: early bailout — healthy doctor execs home screen (no download), unhealthy does not bail, --force skips the bailout. Not run locally (macOS [[ ]] blindspot + spin/sleep/read block without a TTY); assertions written fail-loud against the exact emitted copy for Linux CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(installer): retarget two assertions to the first-run copy Linux CI caught two stale assertions the local checks couldn't (macOS [[ ]] blindspot): - install-bootstrap happy path grepped the old "installer files verified" line, now "All N files intact — nothing was altered". - _pf_storage_type local-fs test grepped the fstype (ext4), which the first-run redesign moved into the log; the visible line is now the clean "Local storage (…)". Assert that instead. Both tests keep their original intent; no code behavior changed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): address Bugbot #341 — real tracebloc cmd, ~/.local/bin bailout probe, exec fallthrough - summary.sh "What's next" said `tb data ingest`, but there is no `tb` on PATH (the binary is `tracebloc`; the CTA two lines below already says `tracebloc`) → use `tracebloc data ingest`. - install.sh bailout probed `command -v tracebloc` before any PATH prepend, so a healthy CLI in ~/.local/bin (the installer's fallback dir, not on a fresh curl|bash PATH) was missed → forced a needless full re-download. Prepend ~/.local/bin first, mirroring provision_client. - install.sh bailout `exec tracebloc || true; exit 0` silently exited 0 if exec failed (bad interpreter / missing exec bit) — healthy machine, but no home screen and no install. Dropped `|| true`; if exec returns (i.e. it failed), print an actionable line and exit 1. Regenerated scripts/manifest.sha256 (gen-manifest.sh --check passes); retargeted the summary.bats assertion to the corrected command. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): redirect bailout CLI stdin away from the curl|bash pipe (Bugbot #341) Bugbot HIGH (learned rule): under `curl … | bash` the healthy-setup bailout ran `tracebloc doctor` and `exec tracebloc` without redirecting stdin, so the child CLI inherited the install pipe as stdin — a CLI that reads stdin could block the bailout or consume the pipe instead of behaving non-interactively. - `tracebloc doctor` (bounded health probe): stdin ← /dev/null (non-interactive). - `exec tracebloc` (interactive home-screen hand-off): stdin ← /dev/tty (the user's real terminal, not the pipe); the exec-failure fallthrough (incl. no controlling terminal to open /dev/tty) keeps the existing "couldn't launch → exit 1" path. bash -n + shellcheck clean. install.sh is the bootstrap (not in the signed manifest), so no manifest change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): probe /dev/tty openability before the bailout exec (fixes #341 CI) c19b51d's `exec tracebloc </dev/tty` is unconditional, but a readable /dev/tty device node with no controlling terminal (CI, detached sessions) still fails to OPEN — the redirect then aborts the exec, the healthy bailout prints "could not launch" + exit 1, and every job that runs install.sh (bats, unit, prereqs, path-persist) goes red. Probe openability first: `</dev/tty` when it opens, else `</dev/null` — still always execs (never the install pipe), so the hand-off and its bats coverage hold. install.sh isn't a hashed sub-script (manifest unchanged). bats install-bootstrap green; shellcheck clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * Merge pull request #344 from tracebloc/test/290-check-drift-cli-invariants test(drift): pin the CLI-assumed chart contract in check-drift.sh * test(e2e-proxy): deflake §4 — retry the squid Service-readiness race + raw log on failure (#346) §4 (the #119 ingestion-egress guard) flaked red on develop (run 29255451968); a re-run of the SAME commit (run 29340987746) passed — a transient, not a #341 regression (#341 never touched this test). Cause: the app pod's section-A curl reaches the in-cluster squid via its Service, but `kubectl rollout status` gates only on the squid pod being Ready, not on its Service endpoints being programmed in kube-proxy. In that window a brand-new pod's first CONNECT to the ClusterIP is refused, and curl gave up before attempting the tunnel (no "Establish HTTP proxy tunnel" line) — so the assertion false-failed. - Add `--retry 5 --retry-connrefused --retry-delay 2` (-m 30) to section A's curl: it rides out the transient connection-refused. A genuine #119 regression still fails all retries, so the guard keeps its teeth. - On the (A) assertion failure, dump the RAW egress-app log + squid pod/endpoints state. This flake surfaced only the filtered B line and a bare "did NOT tunnel"; the raw dump makes any future failure debuggable instead of opaque. bash -n + shellcheck clean. (The k3d e2e can't run locally; CI is the authority.) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): resolve Bugbot findings on #345 (tty handoff, pinned-ref reinstall, jq split) (#347) * fix(installer): resolve Bugbot findings on #345 (tty handoff, pinned-ref reinstall, jq split) Three Cursor Bugbot findings on the develop→main PR: - High — assess.sh `_assess_handoff` ran a bare `tracebloc` with no stdin redirect. Under `curl | bash` that inherits the install pipe, so the interactive home-screen handoff could consume script bytes or block (same class as #341). Redirect </dev/tty when openable, else </dev/null — mirrors the bootstrap hand-off in install.sh; still no `exec` so the EXIT trap runs. - High — install.sh skipped its healthy early-bailout on a pinned REF/BRANCH/unverified request but never told install-k8s.sh's own stop-and-check gate. A pinned-ref re-run would download the new installer and then short-circuit to the home screen, doing nothing. Export TB_FORCE_REINSTALL whenever the bailout is skipped so the downstream gate runs the full flow. - Low — assess.sh `_assess_cluster_servers_running` reintroduced a jq/awk bifurcation; jq is not a guaranteed installer prerequisite (Bugbot #284). Collapse to the single jq-free awk path (the SERVERS column read already there). Adds bootstrap tests asserting the reinstall intent reaches install-k8s.sh; test suites green (assess 31/31, install-bootstrap 15/15). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(supply-chain): regenerate manifest.sha256 for assess.sh change Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): give the healthy-machine hand-off a real terminal on stdout/stderr (#348) Bugbot (Medium) follow-up to #347: main() runs setup_log_file (`exec > >(tee …) 2>&1`) BEFORE the stop-and-check gate, so by the time _assess_handoff runs the shell's stdout/stderr are a pipe to tee, not the terminal. The earlier fix redirected only stdin, so the interactive `tracebloc` home screen still rendered onto the tee pipe instead of a tty (the bootstrap bailout in install.sh avoids this only because it hands off before any tee redirect). Point all three streams at the terminal ($TB_TTY, /dev/tty) when it's openable — bypassing tee for the interactive screen, exactly as the bootstrap does — else fall back to </dev/null and leave stdout/stderr on the pipe (non-interactive/CI). Adopt provision.sh's TB_TTY indirection so the redirect is testable; assess.bats now proves the home screen lands on the terminal (not the pipe) on the openable path and falls back cleanly otherwise. Manifest regenerated. assess.bats 32/32. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): assess gate checks cluster is running before the Helm probe (#349) * fix(installer): check cluster is running before the Helm probe in the assess gate Bugbot (High) follow-up: _assess_classify called detect_installed_client (unbounded `helm list -A` / `helm get values`) BEFORE the cluster-servers-running check. On a stopped cluster (post-reboot or manual stop) the k8s API is down, so Helm hangs — violating the gate's "bounded, never-hang" contract — and, when it finally fails, the machine is mislabeled fresh/cluster-no-release, making the cluster-stopped path effectively unreachable on real re-runs. Reorder so the cheap read-only k3d servers-running probe runs first: a stopped cluster short-circuits to degraded/cluster-stopped without ever touching Helm, and detect_installed_client only ever runs against a live API. Adds an ordering guard test asserting the Helm probe is not invoked on a stopped cluster. Manifest regenerated. assess.bats 33/33. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(e2e-proxy): deflake §4 — retry the app-pod proxy-DNS resolution race The §4 app-pod curl used --retry-connrefused, which retries a refused CONNECT (Service endpoints not yet programmed) but NOT a DNS failure. Under CI load the squid Service name isn't yet in the new pod's resolver when curl runs, so it fails with "Could not resolve proxy" (curl exit 5) — which curl does not retry on its own — and §4 flaked red (PR #349 run 29345383969) even though the squid pod and endpoints were up. Add --retry-all-errors (covers the DNS-resolution error too) and bump to --retry 8. A genuine #119 regression still fails all retries, so the guard keeps its teeth. Test-only; not part of the installer supply-chain manifest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): fail closed on unreadable one-client guard values; fix bootstrap spinner on bash 3.2 (#350) Two Bugbot findings on #345: - Medium (install-k8s.ps1): the Windows one-client-per-machine guard treated an unreadable `helm get values` (fetch failure OR unparsable JSON) for a client-chart release the same as "no client here" — so if the ONLY installed client's values couldn't be read, $existingId stayed empty and the install proceeded, silently overwriting a client it couldn't identify (fail-open). Now record any client release whose clientId we can't read and fail CLOSED: refuse with an actionable message rather than overwrite an unknown client. A parsed release with no clientId (literal `null`) is unchanged — still not a match. Adds Pester coverage for both unreadable paths. - Low (install.sh): the early-bailout spinner stored 3-byte braille frames in a single string and indexed `${frames:i:1}` / `${#frames}` — byte-based on macOS system bash 3.2, so glyphs sliced mid-byte and rendered as garbage. Switch to an array of glyphs indexed by element (matches spin() in lib/common.sh), which is char-correct on 3.2. Manifest regenerated (install-k8s.ps1 hash). install.sh shellcheck clean, .ps1 parses cleanly. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): bound the services progress-bar kubectl calls with --request-timeout Bugbot (Medium): _download_services_progress claims to be bounded/non-blocking, but its two `kubectl get pods` calls omitted --request-timeout. The TB_PULL_TIMEOUT deadline is only checked BETWEEN iterations, so a wedged/ unreachable API makes kubectl block indefinitely — freezing step e's progress bar before the authoritative readiness gate in step f ever runs. Add --request-timeout (default 5s, overridable via TB_PROGRESS_KUBECTL_TIMEOUT) to both calls, mirroring assess.sh's bounded probe — a call that can't reach the API now returns quickly, the loop re-checks the deadline, and step e degrades to the honest "still downloading in the background" line instead of hanging. Manifest regenerated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): proactive hardening sweep (bounded probes, tty handoffs, fail-closed guards) (#352) * fix(installer): bound the services progress-bar kubectl calls with --request-timeout Bugbot (Medium): _download_services_progress claims to be bounded/non-blocking, but its two `kubectl get pods` calls omitted --request-timeout. The TB_PULL_TIMEOUT deadline is only checked BETWEEN iterations, so a wedged/ unreachable API makes kubectl block indefinitely — freezing step e's progress bar before the authoritative readiness gate in step f ever runs. Add --request-timeout (default 5s, overridable via TB_PROGRESS_KUBECTL_TIMEOUT) to both calls, mirroring assess.sh's bounded probe — a call that can't reach the API now returns quickly, the loop re-checks the deadline, and step e degrades to the honest "still downloading in the background" line instead of hanging. Manifest regenerated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): proactive hardening sweep — bounded probes, tty handoffs, fail-closed guards A comprehensive audit of the installer (bash + PowerShell) for the failure classes Bugbot has been surfacing one at a time on #345, fixing the genuine instances in one pass. Bounded probes (no more indefinite hangs against a wedged/unreachable API): - cluster.sh: `kubectl cluster-info` in the _wait_for_api gate (the 60s cap was only re-checked between iterations — a single call could block forever). - summary.sh: cluster-info / get nodes / get pods / logs on the summary + not-ready diagnostic paths. - diagnose.sh: every kubectl call in the support-bundle path (`set +e` stops aborts but NOT hangs); helm calls now run only behind a bounded cluster-info probe (helm has no --request-timeout). - gpu-plugins.sh: `kubectl get nodes` in the verify_gpu poll loop. - common.sh download_with_progress: HEAD probe (-m) + the backgrounded curl (--connect-timeout + --speed-limit/--speed-time stall abort — it was monitored only by `kill -0`, no deadline, no kill). - install.sh: every bootstrap fetch (--connect-timeout/--max-time; retry already present, so a stall becomes retriable). - install-cli.sh: the CLI-installer download (a stall now falls to "install later" instead of hanging the step). curl|bash tty handoffs (stdin is the install pipe; stdout/stderr are the tee): - provision.sh: `tracebloc login` (the credential-mint device flow) now gets the real terminal on all three streams when openable, else </dev/null — same idiom as assess.sh's hand-off. - gpu-nvidia.sh: the reboot `read` read from the EOF pipe with no `|| true`, aborting the whole installer under `set -e` right after a successful driver install; now reads /dev/tty, no-tty => no reboot. - setup-macos.sh: the Docker-arch replace prompt read the pipe (empty answer => meaningless confirm); now reads /dev/tty. Fail-closed guards (a failed check must not read as "safe to proceed"): - install-client-helm.sh detect_installed_client now reports INSTALLED_CLIENT_UNKNOWN=1 when `helm list` FAILS (vs genuinely no releases), and the one-client guard refuses rather than risk overwriting a client it couldn't enumerate. +bats coverage. - install-k8s.ps1 one-client guard: same fix at the `helm list` level (non-zero exit OR non-JSON now fails closed, matching the per-release fix from #350). +Pester coverage. set -e footgun: - common.sh _chart_version: trailing `|| true` so a no-match `grep` (no client release) can't abort callers that assign it under `set -e`. PowerShell 5.1 portability: - install-k8s.ps1: the WSL-path build used a scriptblock `-replace` (PS 6.1+); under Windows PowerShell 5.1 (the bootstrap target) the drive letter wasn't lowercased -> malformed path -> 180s NCT-install timeout. Now -match/$Matches. Deliberately NOT changed (documented): - k3d `cluster create --wait` has no --timeout in EITHER bash or PowerShell — at parity, and the known --wait hang cause (proxy misroute) is already mitigated by the proxy config. Not changing critical-path create behavior speculatively. - The dataset-mount check's inspect-failure no-op is a documented, tested design choice (cluster.bats) — left as-is. - Long tail of Linux-GPU-only driver-download curls (gpu-nvidia/gpu-amd, setup-linux) and misc PowerShell -TimeoutSec: LOW severity, retry-wrapped; left for a follow-up to keep this reviewable. All bash suites green (assess 33, install-client-helm 54, cluster 27, provision, summary, install-cli, bootstrap, preflight); shellcheck --severity=error clean; install-k8s.ps1 parses cleanly; manifest regenerated. Two pre-existing local-env bats failures (_extract_yaml_value '' escape; validate_config) are unrelated and also fail on develop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(installer): fail closed on unknown helm state in the #303 pre-provision guard Bugbot (High) on the sweep: the sweep taught detect_installed_client to signal INSTALLED_CLIENT_UNKNOWN=1 on a failed `helm list` and wired it into the Helm-step one-client guard — but provision_client's #303 pre-flight still only checked INSTALLED_CLIENT_NS. A failed enumeration leaves both globals empty, so provisioning continued to `client create` and could register a dashboard client the later Helm guard then refuses to install — the exact orphan the pre-flight exists to prevent. Fail closed on INSTALLED_CLIENT_UNKNOWN right after detect_installed_client, before any mint — same signal the Helm-step guard keys on. +bats coverage (unknown state refuses before `client create`, no orphan). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * Merge pull request #353 from tracebloc/fix/bugbot-345-detect-values-failopen fix(installer): bash guard fails closed on unreadable client values (PS parity) --------- Co-authored-by: lukasWuttke <54042461+LukasWodka@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Arturo Peroni <arturo@tracebloc.io>
…from timezone (#356) * Merge pull request #355 from tracebloc/feat/provision-drop-location-prompt feat(installer): stop prompting for location during register — auto-derive from timezone (#354) * fix(installer): attribute rejected-zone hint to its real source (Bugbot #356) (#357) * fix(installer): attribute rejected-zone hint to its real source (Bugbot #356) The invalid-location failure branch always told the operator the zone was auto-derived from the timezone and pointed them at TRACEBLOC_CLIENT_LOCATION as the fix. When the rejected value actually came FROM that env override, the hint read as "your override was ignored" — the opposite of what happened. Track whether the location was pinned via TRACEBLOC_CLIENT_LOCATION ("env") or timezone-derived ("auto") and pass the source into _report_create_failure so the fix hint names the real origin. Defaults to "auto" when the caller omits it. Adds three unit tests covering both sources plus the default. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(installer): refresh manifest checksum for provision.sh (R8) The rejected-zone-hint edit changed scripts/lib/provision.sh; regenerate scripts/manifest.sha256 so the supply-chain "manifest is current" check passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
chore(chart): bump client 1.9.4 → 1.9.5 (version + appVersion)
main - release
Release develop → main
release-train: develop -> staging
release-train: staging -> main
release-train: develop -> staging
release-train: develop -> staging
release-train: develop -> staging
release-train: develop -> staging
release-train: develop -> staging
release-train: develop -> staging
release-train: develop -> staging
release-train: staging -> main
release-train: develop -> staging
release-train: develop -> staging
release-train: staging -> main
release-train: develop -> staging
release-train: develop -> staging
release-train: staging -> main
release-train: develop -> staging
release-train: develop -> staging
release-train: develop -> staging
release-train: develop -> staging
release-train: staging -> main
release-train: develop -> staging
release-train: develop -> staging
release-train: staging -> main
…pying it CI parity. Two jobs restated targets the Makefile already declares -- and one of them had ALREADY DRIFTED, in the direction nobody notices. LINT. This job spelled out the shellcheck file list inline while the Makefile kept the same list in SHELLCHECK_FILES. Measured: the Makefile carries 19 entries, this file carried 9. Ten scripts were shellchecked on a contributor machine and NOT at the merge gate: gen-manifest.sh check-facts.sh check-style.sh lib/*.sh tests/check-drift.sh tests/e2e-full-seal.sh tests/e2e-journey.sh tests/path-persist.sh tests/chart-env-vocabulary.sh tests/env-vocabulary-agreement.sh gen-manifest.sh is the installer integrity-manifest generator, so a shell defect there could not be caught by this gate. `make lint` is green across all 19 on this tree, so arming the full list imports no backlog. HELM LINT. The values-file loop, `helm lint --strict ./ingestor` and both vocabulary scripts were verbatim copies. `env-vocabulary-agreement.sh` exists to prove the four CLIENT_ENV declarations agree with each other (backend#1729 sweep 5) -- a check about "these declarations must not drift" being itself declared twice is the joke version, and a third vocabulary script added to the Makefile alone would leave this gate silently not running it. The apt install of shellcheck stays: it bootstraps the runner, it is not a duplicated command. Job names untouched. `Lint` and `Unit tests` are required status checks on main, matched by name. Verified: make lint, make helm-lint and make helm-vocab all exit 0 on this tree; actionlint clean on both files. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Resolves the helm-ci.yaml conflict with #715, which moved the CLIENT_ENV vocabulary-agreement guard out of `Helm lint` into drift-checks.yaml's required `Source-of-truth drift` job. Calling `make helm-vocab` from the lint job would have quietly put it back, so the guard moves to `make drift` (the target that mirrors the drift gate) and `helm-vocab` is now the chart vocabulary only. `make check` runs both, so the local tier is unchanged. Bugbot, two findings on b5c21fe: - Helm setup removed from lint job. Correct: dropping the azure/setup-helm v3.15.4 pin left both make targets running against whatever helm the runner image ships, and chart-env-vocabulary.sh branches on the helm version (its helper-backstop cases self-skip below 3.16), so the pin decides which assertions run. Installing helm is environment bootstrap, the same class as the shellcheck apt install this PR deliberately kept. Restored, with the same action + pin drift-checks.yaml uses. - Makefile missing from path filters. Correct: the lint job now executes the Makefile, so a Makefile-only edit to those targets has to be able to start this workflow. Added to both `paths:` lists. standard-checks.yml needs no equivalent -- it is deliberately not path-filtered at all, being a required check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 34b8f6d. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of backend#1606, under epic backend#1680. Completes the CI-parity set.
The lists had already drifted — and CI was the weaker side
The
Lintjob spelled out its shellcheck file list inline, while the Makefile kept the same list inSHELLCHECK_FILES. Measured:SHELLCHECK_FILESstandard-checks.ymlinlineTen scripts were shellchecked on a contributor's machine and not at the merge gate:
gen-manifest.sh·check-facts.sh·check-style.sh·lib/*.sh·tests/check-drift.sh·tests/e2e-full-seal.sh·tests/e2e-journey.sh·tests/path-persist.sh·tests/chart-env-vocabulary.sh·tests/env-vocabulary-agreement.shgen-manifest.shis the installer's integrity-manifest generator — a shell defect there could not be caught by this gate.make lintis green across all 19 on this tree, so arming the full list imports no backlog.Helm lint
The
client/ci/*-values.yamlloop,helm lint --strict ./ingestor, and both vocabulary scripts were verbatim copies ofhelm-lint+helm-vocab.Worth naming why this one bites:
env-vocabulary-agreement.shexists to prove the fourCLIENT_ENVdeclarations agree with each other (backend#1729 sweep 5). A check about "these declarations must not drift" being itself declared twice is the joke version — and a third vocabulary script added to the Makefile alone would leave this gate silently not running it.What stays
The apt install of shellcheck — it bootstraps the runner, it is not a duplicated command. Same call as release-train#80's venv step.
Job names untouched
LintandUnit testsare required status checks on main, matched by name.Verification
make lint,make helm-lint,make helm-vocaball exit 0 on this tree;actionlintclean on both files.Note
Low Risk
CI wiring and Makefile target boundaries only; strengthens lint coverage and preserves which job runs the vocabulary-agreement guard.
Overview
Completes backend#1606 CI parity by having
standard-checksandhelm-ciinvoke Makefile targets instead of duplicating shell commands.The required
Lintjob now runsmake lint, so merge gating uses the sameSHELLCHECK_FILESlist as localmake check(19 scripts instead of the workflow’s stale inline list of 9).helm-ci’sHelm lintjob runsmake helm-lint helm-vocabafter the existing Helm pin;Makefileis added to that workflow’spathsso edits to those targets still trigger the gate.env-vocabulary-agreement.shmoves fromhelm-vocabintomake driftonly, so routing helm CI through the Makefile does not put that guard back on the optional Helm lint job after #715 moved it to the required drift gate.make checkstill runsdriftandhelm-vocabseparately, so local behavior stays the same.Reviewed by Cursor Bugbot for commit 34b8f6d. Bugbot is set up for automated code reviews on this repo. Configure here.