Skip to content

Fix ENOSPC empty media on large ISOs; harden stage 3 build - #11

Open
isss802 wants to merge 1 commit into
kitknox:mainfrom
isss802:fix/large-iso-build-space
Open

Fix ENOSPC empty media on large ISOs; harden stage 3 build#11
isss802 wants to merge 1 commit into
kitknox:mainfrom
isss802:fix/large-iso-build-space

Conversation

@isss802

@isss802 isss802 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Problem

Large-ISO installs (e.g. English Windows 11) hang at boot. The console sits at

Booting from Hard Disk...

indefinitely — CPU pinned at 100%, disk I/O zero — and never enters Windows Setup.

Root cause: stage3 runs distrobuilder repack-windows on the ~30 GB build root. For the English Windows 11 ISO that root cannot hold the source (~8.5 GB) plus distrobuilder's extraction overlay plus the output ISO (~8.5 GB) at once, so ISO generation fails with ENOSPC (genisoimage exit 28) and leaves a truncated windows_virtio.iso. The only guard is [ -f windows_virtio.iso ] (existence), which the truncated file passes, so WoeUSB is handed a corrupt ISO, writes an empty media partition, and the script reboots into an unbootable disk. Smaller ISOs (Windows Server 2022, ~5 GB) fit in 30 GB, which is why only large ISOs are affected.

Confirmed by rescue-mounting a hung instance's disk: the 8 GB media partition held only autounattend.xml — no bootmgr, no sources/install.wim. SeaBIOS finds the boot-flagged partition, prints "Booting from Hard Disk…", finds no bootmgr, and spins.

Fix

Run the heavy repack on the 160 GB target disk (/dev/sdb), which is unused until it is partitioned later in the same function:

  • Mount /dev/sdb at /mnt/work; put the source ISO, distrobuilder --cache-dir, the output ISO, and TMPDIR there.
  • Validate the output, copy it back to the root, then hard-gate the unmount before repartitioning sdb.
  • Enlarge the media partition 8 GB → 12 GB (the repacked English media is ~8.5 GB).

Harden stage3 so a broken build fails loudly instead of rebooting into a dead disk:

  • Halt (no reboot) on any failure, leaving /root/BUILD_FAILED and the build environment alive for inspection.
  • Validate windows_virtio.iso size and contents (sources/install.wim, bootmgr).
  • Check WoeUSB's exit code; after WoeUSB, verify the media partition actually contains bootmgr + install.wim.
  • Verify the MBR bootstrap is non-empty — an empty MBR reproduces the exact "Booting from Hard Disk…" hang even when the files are correct.
  • Move partprobe / udevadm settle before formatting (it was racing device-node creation); size-check the staged ISO copy; install libarchive-tools for a reliable ISO content listing.

Testing

Deployed on Linode (jp-tyo-3, g6-standard-4).

  • Before: English Windows 11 25H2 → SeaBIOS "Booting from Hard Disk…" hang; media partition empty.
  • After: English Windows 11 25H2 installs unattended and reaches RDP — raw X.224 Connection Confirm 03 00 00 13 0e d0 ….
  • The same instrumentation is what surfaced the original ENOSPC (exit 28) during diagnosis (the build halted with a clear message instead of silently shipping empty media).
  • Control: Windows Server 2022 (small ISO) still builds end-to-end.

🤖 Generated with Claude Code

## Problem

Large-ISO installs (e.g. English Windows 11) hang at boot. The console sits at

```
Booting from Hard Disk...
```

indefinitely — CPU pinned at 100%, disk I/O zero — and never enters Windows
Setup.

Root cause: `stage3` runs `distrobuilder repack-windows` on the **~30 GB build
root**. For the English Windows 11 ISO that root cannot hold the source
(~8.5 GB) **plus** distrobuilder's extraction overlay **plus** the output ISO
(~8.5 GB) at once, so ISO generation fails with **ENOSPC** (`genisoimage` exit
28) and leaves a **truncated** `windows_virtio.iso`. The only guard is
`[ -f windows_virtio.iso ]` (existence), which the truncated file passes, so
WoeUSB is handed a corrupt ISO, writes an **empty media partition**, and the
script reboots into an unbootable disk. Smaller ISOs (Windows Server 2022,
~5 GB) fit in 30 GB, which is why only large ISOs are affected.

Confirmed by rescue-mounting a hung instance's disk: the 8 GB media partition
held only `autounattend.xml` — no `bootmgr`, no `sources/install.wim`. SeaBIOS
finds the boot-flagged partition, prints "Booting from Hard Disk…", finds no
`bootmgr`, and spins.

## Fix

Run the heavy repack on the **160 GB target disk** (`/dev/sdb`), which is unused
until it is partitioned later in the same function:

- Mount `/dev/sdb` at `/mnt/work`; put the source ISO, distrobuilder
  `--cache-dir`, the output ISO, and `TMPDIR` there.
- Validate the output, copy it back to the root, then hard-gate the unmount
  before repartitioning `sdb`.
- Enlarge the media partition 8 GB → 12 GB (the repacked English media is
  ~8.5 GB).

Harden `stage3` so a broken build fails **loudly** instead of rebooting into a
dead disk:

- **Halt (no reboot)** on any failure, leaving `/root/BUILD_FAILED` and the
  build environment alive for inspection.
- Validate `windows_virtio.iso` size and contents (`sources/install.wim`,
  `bootmgr`).
- Check WoeUSB's exit code; after WoeUSB, verify the media partition actually
  contains `bootmgr` + `install.wim`.
- Verify the **MBR bootstrap is non-empty** — an empty MBR reproduces the exact
  "Booting from Hard Disk…" hang even when the files are correct.
- Move `partprobe` / `udevadm settle` **before** formatting (it was racing
  device-node creation); size-check the staged ISO copy; install
  `libarchive-tools` for a reliable ISO content listing.

## Testing

Deployed on Linode (`jp-tyo-3`, `g6-standard-4`).

- **Before:** English Windows 11 25H2 → SeaBIOS "Booting from Hard Disk…" hang;
  media partition empty.
- **After:** English Windows 11 25H2 installs unattended and reaches RDP — raw
  X.224 Connection Confirm `03 00 00 13 0e d0 …`.
- The same instrumentation is what surfaced the original ENOSPC (exit 28) during
  diagnosis (the build halted with a clear message instead of silently shipping
  empty media).
- **Control:** Windows Server 2022 (small ISO) still builds end-to-end.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant