feat: extract build infrastructure into standalone scripts by wdconinc · Pull Request #227 · eic/containers

wdconinc · 2026-04-06T18:31:31Z

Summary

Add build-base.sh and build-eic.sh as standalone scripts that encapsulate the full docker buildx build invocations previously embedded in .gitlab-ci.yml and .github/workflows/build-push.yml. These scripts are now the single source of truth for build logic — they are called by the CI and can be run directly by users for local builds.

Motivation

The .gitlab-ci.yml previously contained 130+ lines of inline docker buildx build commands with ~20–30 --build-arg flags each. This made local builds difficult because:

The embedded scripts couldn't be run directly
The docs/building-locally.md "Full Build Script" was missing ~15 --build-arg entries (SHAs, cherry-picks, duplicate allowlists, etc.)
Any change to build logic required updating both CI and docs separately

Changes

New files

build-base.sh — builds debian_stable_base, cuda_devel, cuda_runtime
build-eic.sh — builds any EIC environment (ci, xl, cuda, dbg, etc.)

CI/local mode detection

Scripts auto-detect their context via CI_REGISTRY:

Feature	CI mode	Local mode
Output	`--push`	`--load`
Build cache write	`--cache-to` to CI registry	omitted
Build cache read	CI + ghcr.io registries	public ghcr.io only
`mirrors.yaml`	full template with credentials	public-only, no credentials
Image tags	`INTERNAL_TAG`, `EXPORT_TAG`	`--tag local`

Local usage

# Build base image
bash scripts/build-base.sh --jobs 8

# Build EIC XL environment
bash scripts/build-eic.sh --env xl --jobs 8

# Build EIC CI environment (faster)
bash scripts/build-eic.sh --env ci

.gitlab-ci.yml

The base and eic job scripts are each reduced to two lines:

script:
  - apk add git
  - bash scripts/build-<x>.sh

All CI matrix variables (BUILD_IMAGE, ENV, BUILD_TYPE, etc.) flow through naturally as environment variables.

.github/workflows/build-push.yml

The base and eic jobs are reduced to, essentially:

      - name: Build and push
        env:
          BUILD_IMAGE: ${{ matrix.BUILD_IMAGE }}
          BASE_IMAGE: ${{ matrix.BASE_IMAGE }}
        run: bash scripts/build-base.sh

docs/building-locally.md

Replaced the incomplete inline script with references to build-base.sh and build-eic.sh, updated all examples and troubleshooting steps.

Copilot

Pull request overview

This PR extracts the long docker buildx build invocations from .gitlab-ci.yml into two standalone root-level scripts (build-base.sh, build-eic.sh) intended to be the single source of truth for both CI and local builds, and updates local build documentation to use these scripts.

Changes:

Add build-base.sh to build base images (Debian stable base and CUDA base/runtime).
Add build-eic.sh to build EIC environment images with CI/local mode behavior.
Update .gitlab-ci.yml and docs/building-locally.md to delegate build logic to the new scripts.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

File	Description
`build-base.sh`	New script encapsulating the base image `docker buildx build` logic.
`build-eic.sh`	New script encapsulating the EIC environment image build logic (tagging, caching, secrets, mirrors).
`.gitlab-ci.yml`	Replaces large inline build command blocks with calls to the new scripts.
`docs/building-locally.md`	Updates local build instructions to use the new scripts and revises build-arg guidance.

GITHUB_BASE_REF is empty for push events, so CI_DEFAULT_BRANCH_SLUG was always falling back to the hardcoded string 'master', degrading cache hits on repos whose default branch is 'main' or anything else. Pass DEFAULT_BRANCH from the workflow (github.event.repository.default_branch) and use it as an intermediate fallback before 'master': CI_DEFAULT_BRANCH_SLUG=$(slugify "${GITHUB_BASE_REF:-${DEFAULT_BRANCH:-master}}") On PR events GITHUB_BASE_REF is still used (the PR target branch). On push events GITHUB_BASE_REF is empty and DEFAULT_BRANCH takes over. The 'master' final fallback covers local runs with no CI env set. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Two robustness issues in the mirrors.yaml generation: 1. sed substitution was not escaping replacement values, so registry URLs or paths containing '&', '\', or '|' could corrupt the output. Add sed_escape() helper that escapes those characters before injection. 2. mirrors.yaml was written into the source tree (SCRIPT_DIR), leaving a dirty working tree for local users and risking collisions when multiple builds run from the same checkout concurrently. Switch to mktemp under TMPDIR and register a trap to clean it up on exit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The fallback is triggered when either BUILDER_IMAGE or RUNTIME_IMAGE (or both) are absent locally, but the old message only named BUILDER_IMAGE, leaving users guessing which image was actually missing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

build-eic.sh only passes CI_REGISTRY_USER/CI_REGISTRY_PASSWORD/ GITHUB_REGISTRY_USER/GITHUB_REGISTRY_TOKEN as secrets in CI mode. Local builds omit them, but the Dockerfile unconditionally declared those secret mounts as required, causing 'secret not found' failures for anyone running a local build. Add required=false to all four credential secret mounts in the three RUN installation steps. The mirrors secret is always provided (via mktemp in both CI and local mode) and remains required. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* build-eic: support comma-separated --build-type (default,nightly) Build both default and nightly images in a single sequential invocation so that the four shared Docker stages (builder/runtime concretization and installation of the default spack environment) are reused from BuildKit's layer cache. Cuts redundant Spack compilation for nightly builds that follow a default build on the same runner. Key changes: - build-eic.sh: change BUILD_TYPE default to "default,nightly"; split on comma; validate each token; loop over types – resolving EIC package SHAs and constructing the docker-buildx command per iteration. Shared setup (benchmark SHAs, mirrors.yaml, SPACK_DUPLICATE_ALLOWLIST, ARCH) runs once before the loop. Metadata files are written as ${METADATA_FILE%.json}-<build_type>.json; logs as build-<build_type>.log. - .gitlab-ci.yml: restructure eic parallel matrix from 16 to 10 jobs by using BUILD_TYPE="default,nightly" as a plain string (not a list dimension) for environments that previously had separate default and nightly rows. Drop ${BUILD_TYPE} from METADATA_FILE (script now appends it). Update .build artifacts glob to build-*.log. Fix .nightly rules so that combined jobs override BUILD_TYPE to "default" on stable branches (instead of matching only the now-absent BUILD_TYPE=="nightly" string), while nightly-only jobs (ci_without_acts) retain when:never semantics. - .github/workflows/build-push.yml: set BUILD_TYPE="default,nightly" for ci and xl matrix entries. Export one digest file per build type and upload as separate artifacts. Add BUILD_TYPE dimension to eic-manifest matrix; update artifact download pattern; add build-type-aware tags. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

- BASE_IMAGE: debian:trixie-slim - update shellcheck script pattern - nproc -> getconf _NPROCESSORS_ONLN - portable sed

Copilot

Pull request overview

Copilot reviewed 8 out of 11 changed files in this pull request and generated 4 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

github-actions · 2026-07-05T17:27:51Z

Capybara summary for PR 227

wdconinc · 2026-07-08T23:53:36Z

@copilot Review this PR in light of the other ways in which jobs are run on the gitlab instance, besides the CI that is exercised here and there in pushes. In particular, does this PR affect:

scheduled runs that define NIGHTLY,
triggered runs from other gitlab repositories that define VERSION,
tags that exercise the version logic in .gitlab-ci.yml.

Copilot · 2026-07-08T23:56:47Z

@copilot Review this PR in light of the other ways in which jobs are run on the gitlab instance, besides the CI that is exercised here and there in pushes. In particular, does this PR affect:

sched...

Reviewed all three scenarios against the current scripts/build-eic.sh and .gitlab-ci.yml:

Scheduled runs that define NIGHTLY

NIGHTLY is not referenced anywhere in build-eic.sh. Nightly building is controlled by BUILD_TYPE (the matrix sets BUILD_TYPE: "default,nightly" for main environments), while NIGHTLY only gates tagging inside eic-manifest (lines 501–505: [ -n "${NIGHTLY}" ] decides whether to apply NIGHTLY_TAG). This tagging logic was already in the manifest job and is unchanged. The PR has no impact on scheduled runs.

Triggered runs that define VERSION

VERSION is consumed by the version job, which writes EXPORT_TAG (= VERSION for external triggers) to the dotenv artifact. build-eic.sh uses EXPORT_TAG only for the informational EIC_CONTAINER_VERSION build arg (${EXPORT_TAG}-${build_type}-${SHA}). Actual registry tagging with EXPORT_TAG remains entirely in eic-manifest. The .nightly rules continue to filter non-ci environments for external triggers ($ENV != "ci" && $CI_PIPELINE_SOURCE == "trigger"). No impact.

One minor observation: for branches where EXPORT_TAG="" (non-master, no version export), EIC_CONTAINER_VERSION gets a leading dash (e.g. -default-abc1234). This is a cosmetic label issue only for non-exported branches, and those images are not published, so no functional impact.

Tags (v1.2.3 format)

The version job's tag-parsing logic (lines 151–158) is unchanged, including the verification that the epic@${GEOM_VERSION} package exists in the spack.yaml files. The .nightly rules override BUILD_TYPE: "default" for commit tags (line 297–300), so build-eic.sh receives BUILD_TYPE=default and builds only the default image — correct for release tags. EIC_CONTAINER_VERSION becomes 1.2.3-default-${SHA}, as expected. No impact.

Summary: The PR does not affect any of these three scenarios. NIGHTLY, VERSION, and tag version logic all live in the version job and eic-manifest (unchanged), while build-eic.sh focuses solely on the docker buildx build invocation, receiving already-resolved variables via the CI dotenv artifact.

Copilot AI review requested due to automatic review settings April 6, 2026 18:31

Copilot started reviewing on behalf of wdconinc April 6, 2026 18:32 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

Comment thread build-eic.sh Outdated

Comment thread build-eic.sh Outdated

Comment thread build-eic.sh Outdated

Comment thread build-base.sh

Comment thread build-base.sh Outdated

Comment thread docs/building-locally.md

Comment thread docs/building-locally.md

Copilot AI review requested due to automatic review settings April 6, 2026 19:57