Skip to content

fix(config): disable COMPAT_BRK so randomize_va_space defaults to 2#7

Merged
AprilNEA merged 1 commit into
masterfrom
fix/disable-compat-brk
Jun 10, 2026
Merged

fix(config): disable COMPAT_BRK so randomize_va_space defaults to 2#7
AprilNEA merged 1 commit into
masterfrom
fix/disable-compat-brk

Conversation

@PeronGH

@PeronGH PeronGH commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Problem

CONFIG_COMPAT_BRK=y (the kernel default) forces kernel.randomize_va_space=1
at boot — mmap/stack/vDSO are randomized but brk/heap is not. FEX-Emu's
x86-64 allocator can't lay out its virtual address space under =1 and fails
non-deterministically with Failed to map VMA region, so docker run --platform linux/amd64 crashes (SIGSEGV).

Fix

Disable COMPAT_BRK so the kernel boots with randomize_va_space=2 (full
ASLR), matching what OrbStack and standard distros ship. One line.

Validation

Built on 6.12.11 arm64. With this kernel, FEX runs amd64 deterministically:

$ docker --context arcbox run --rm --platform linux/amd64 alpine uname -m
x86_64        # 10/10

Before: ~0/10 (Failed to map VMA region / SIGSEGV). Isolated by toggling
randomize_va_space at runtime — 1 fails, 2 works — on both the 52-bit
(LPA2) and a 48-bit kernel, so VA width and page size were red herrings; this is
the sole fix.

CONFIG_COMPAT_BRK=y makes the kernel boot with randomize_va_space=1
(brk/heap not randomized). FEX's x86-64 JIT allocator relies on full
ASLR to place its managed VA region; under randomize_va_space=1 it fails
with "Failed to map VMA region" and linux/amd64 containers crash. The
kernel OrbStack ships disables COMPAT_BRK (randomize_va_space=2) and runs
FEX cleanly; matching that here fixes amd64 emulation. No other kernel
change (VA width, SVE/SME) is needed.
Copilot AI review requested due to automatic review settings June 9, 2026 13:32
@PeronGH PeronGH requested a review from AprilNEA June 9, 2026 13:32

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Disables CONFIG_COMPAT_BRK in the ArcBox arm64 kernel config so the kernel boots with kernel.randomize_va_space=2 (full ASLR), addressing non-deterministic amd64 emulation failures (e.g., FEX-Emu under docker run --platform linux/amd64) caused by heap/brk not being randomized under the default =1 behavior.

Changes:

  • Disable CONFIG_COMPAT_BRK in the arm64 kernel config to allow full ASLR (randomize_va_space=2) by default.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@AprilNEA AprilNEA merged commit 4c40116 into master Jun 10, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants