fix(service)!: make DOCKER_HOST module-owned; require uid

CHANGELOG.md

@@ -20,6 +20,7 @@ module follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - The apply script now installs on any declared-standalone host (`standalone.manage: true`), not only with the self-update loop, so manual applies and the loop share the same single apply command; enabling the loop on a host not declared standalone fails at compile time.
 - Freshly provisioned hosts now get a 165536-wide subordinate-ID range, which satisfies the ID-mapping requirement of nested rootless BuildKit builds.
 - Subordinate UID/GID ranges are now provisioned by `rootless_docker.manage`, together with the other rootless-Docker prerequisites, instead of by the runner-account toggle: rootless Docker can now be brought up on a host whose runner account is owned by another system. An existing range entry is never overwritten, and `runner_account.manage` keeps owning the group, user, and home.
+- The managed runner service's `DOCKER_HOST` is now module-owned and derived from `runner_account.uid`; `runner_service.environment` carries additional variables only and rejects a `DOCKER_HOST` line, and a managed service requires `runner_account.uid`.
 
 ### Removed
 
@@ -30,6 +31,7 @@ module follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Fixed
 
+- Clearing `runner_service.environment` through data can no longer leave the runner manager without `DOCKER_HOST`; the derived socket is always present.
 - With `packages.sources.manage` on, repeated applies stop churning apt: an unchanged repository signing key is now a true no-op (no keyring rewrite, no needless `apt-get update`), while a genuine key rotation is still picked up from the vendor's rolling key endpoint and refreshes the apt index.
 
 ## [1.0.0] - 2026-07-09

README.md

@@ -178,9 +178,9 @@ surfaces as a raw Puppet or host error rather than the module's clear preflight
   newer (jammy ships 5.15). On older kernels the fallback is `fuse-overlayfs` [\[17\]](#ref-17)
   (kernel 4.18 or newer, plus the `fuse-overlayfs` package).
 - **Daemon connection (`DOCKER_HOST`):** [\[18\]](#ref-18) Required for every rootless job: it is how the runner
-  manager reaches the rootless daemon to create job containers. Set
-  `DOCKER_HOST=unix:///run/user/<uid>/docker.sock` in the runner service environment, equivalently
-  the `host` setting under `[runners.docker]`. This is distinct from bind-mounting the socket into a
+  manager reaches the rootless daemon to create job containers. Where the module manages the runner
+  service it sets this automatically, deriving `DOCKER_HOST=unix:///run/user/<uid>/docker.sock` from
+  `runner_account.uid`. This is distinct from bind-mounting the socket into a
   job (`socket_mount`), which only jobs that drive Docker themselves need (for example
   `docker buildx`, whose default driver also builds through the daemon), not the runner to start
   containers.
@@ -333,10 +333,12 @@ With `runner_service.manage` on, the module owns the `gitlab-runner` system serv
 systemd drop-in for it (`/etc/systemd/system/gitlab-runner.service.d/10-rootless.conf`), and
 the mode on `/etc/gitlab-runner` so the privilege-dropped manager can read its own config. The
 drop-in runs the manager privilege-dropped as the runner account — that is the module's
-posture, not a knob — with `DOCKER_HOST` pointed at the derived rootless docker socket;
-`runner_service.environment` overrides the environment lines rendered into the drop-in, and
-`runner_service.timeout_stop_sec` sets the graceful-drain window
-([Restarts and graceful shutdown](#restarts-and-graceful-shutdown)).
+posture, not a knob — with `DOCKER_HOST` derived from `runner_account.uid`, so a managed
+service requires that uid. `DOCKER_HOST` is module-owned and the same socket the healthcheck and
+`socket_mount` use; `runner_service.environment` adds any further `Environment=` lines alongside
+it (a `DOCKER_HOST` line there fails the compile; a runner whose jobs need a different daemon uses
+the per-runner `host` key instead). `runner_service.timeout_stop_sec` sets the graceful-drain
+window ([Restarts and graceful shutdown](#restarts-and-graceful-shutdown)).
 
 #### `standalone`
 

REFERENCE.md

@@ -130,9 +130,10 @@ of `config.toml`. Each entry is a hash; recognised keys:
 key `MaxUploadedArchiveSize` (Integer, default 0)),
 `allowed_images` (Array[String]), `allowed_pull_policies` (Array[String]).
 Tokens are merged in from `runner_tokens[token_key]` and must not appear
-here. `host` is only for an externally managed daemon at a non-derived
-path: where the module manages the runner service, the drop-in's
-`DOCKER_HOST` already points the executor at the derived rootless socket.
+here. `host` is a passthrough for GitLab Runner's own `[runners.docker]`
+host setting, pointing one runner's jobs at a specific daemon socket; the
+managed runner service's own `DOCKER_HOST` is derived from the uid and set
+independently of it.
 
 ##### <a name="-rootless_gitlab_runner--runner_defaults"></a>`runner_defaults`
 
@@ -221,9 +222,9 @@ configuration file, its directory, and the account's systemd user tree.
 something a module can sensibly invent. It derives the rootless runtime
 paths (`/run/user/<uid>`, the docker socket) and is enforced on the user
 when `runner_account.manage` is on; the apply fails loud when it is unset
-but needed — with `runner_account.manage`, `rootless_docker.manage` or
-`standalone.self_update.manage` on, or to derive the docker socket path
-for a `socket_mount` runner.
+but needed — with `runner_account.manage`, `rootless_docker.manage`,
+`runner_service.manage` or `standalone.self_update.manage` on, or to derive
+the docker socket path for a `socket_mount` runner.
 * **:home** `Stdlib::Absolutepath`: Home directory of the runner account. Default `/home/gitlab-runner`.
 
 ##### <a name="-rootless_gitlab_runner--packages"></a>`packages`
@@ -312,11 +313,13 @@ Options:
 * **:manage** `Boolean`: Whether to manage the runner system service, its privilege-drop systemd
 drop-in, and the configuration directory's mode so a privilege-dropped
 manager can traverse to its configuration file. Default false.
-* **:environment** `Optional[Array[Pattern[/\A[^\r\n]+\z/]]]`: `Environment=` lines (KEY=value) rendered into the service drop-in. When
-unset, defaults to pointing DOCKER_HOST at the rootless docker socket
-derived from `runner_account.uid`. The `Pattern` enforces the type: each
-line must be non-empty and single-line — a value containing a newline is
-rejected, so it cannot inject an extra systemd directive into the drop-in.
+* **:environment** `Optional[Array[Pattern[/\A[^\r\n]+\z/]]]`: Additional `Environment=` lines (KEY=value) rendered into the service
+drop-in, alongside the module-owned `DOCKER_HOST` line. `DOCKER_HOST` itself
+is derived from `runner_account.uid` and cannot be set here: a `DOCKER_HOST`
+line fails the compile, and a runner whose jobs need a different daemon uses
+the per-runner `host` key. The `Pattern` enforces the type: each line must be
+non-empty and single-line, so a value containing a newline is rejected and
+cannot inject an extra systemd directive into the drop-in.
 * **:timeout_stop_sec** `Optional[Variant[Integer[0], String[1]]]`: `TimeoutStopSec=` written into the manager service drop-in — how long
 systemd waits for a graceful drain before escalating to `SIGKILL`. Unset
 by default, so systemd's `DefaultTimeoutStopSec` (typically 90s) applies;

manifests/init.pp

@@ -69,9 +69,10 @@
 #   key `MaxUploadedArchiveSize` (Integer, default 0)),
 #   `allowed_images` (Array[String]), `allowed_pull_policies` (Array[String]).
 #   Tokens are merged in from `runner_tokens[token_key]` and must not appear
-#   here. `host` is only for an externally managed daemon at a non-derived
-#   path: where the module manages the runner service, the drop-in's
-#   `DOCKER_HOST` already points the executor at the derived rootless socket.
+#   here. `host` is a passthrough for GitLab Runner's own `[runners.docker]`
+#   host setting, pointing one runner's jobs at a specific daemon socket; the
+#   managed runner service's own `DOCKER_HOST` is derived from the uid and set
+#   independently of it.
 # @param runner_defaults
 #   Hash merged under every `runners` entry (`$runner_defaults + $runner`,
 #   keys set on the entry win), so multi-runner data does not repeat `url`,
@@ -123,9 +124,9 @@
 #   something a module can sensibly invent. It derives the rootless runtime
 #   paths (`/run/user/<uid>`, the docker socket) and is enforced on the user
 #   when `runner_account.manage` is on; the apply fails loud when it is unset
-#   but needed — with `runner_account.manage`, `rootless_docker.manage` or
-#   `standalone.self_update.manage` on, or to derive the docker socket path
-#   for a `socket_mount` runner.
+#   but needed — with `runner_account.manage`, `rootless_docker.manage`,
+#   `runner_service.manage` or `standalone.self_update.manage` on, or to derive
+#   the docker socket path for a `socket_mount` runner.
 # @option runner_account [Stdlib::Absolutepath] :home
 #   Home directory of the runner account. Default `/home/gitlab-runner`.
 # @param packages
@@ -174,11 +175,13 @@
 #   drop-in, and the configuration directory's mode so a privilege-dropped
 #   manager can traverse to its configuration file. Default false.
 # @option runner_service [Optional[Array[Pattern[/\A[^\r\n]+\z/]]]] :environment
-#   `Environment=` lines (KEY=value) rendered into the service drop-in. When
-#   unset, defaults to pointing DOCKER_HOST at the rootless docker socket
-#   derived from `runner_account.uid`. The `Pattern` enforces the type: each
-#   line must be non-empty and single-line — a value containing a newline is
-#   rejected, so it cannot inject an extra systemd directive into the drop-in.
+#   Additional `Environment=` lines (KEY=value) rendered into the service
+#   drop-in, alongside the module-owned `DOCKER_HOST` line. `DOCKER_HOST` itself
+#   is derived from `runner_account.uid` and cannot be set here: a `DOCKER_HOST`
+#   line fails the compile, and a runner whose jobs need a different daemon uses
+#   the per-runner `host` key. The `Pattern` enforces the type: each line must be
+#   non-empty and single-line, so a value containing a newline is rejected and
+#   cannot inject an extra systemd directive into the drop-in.
 # @option runner_service [Optional[Variant[Integer[0], String[1]]]] :timeout_stop_sec
 #   `TimeoutStopSec=` written into the manager service drop-in — how long
 #   systemd waits for a graceful drain before escalating to `SIGKILL`. Unset
@@ -311,13 +314,13 @@
   # Defaults merged under every runner entry; keys set on the entry win.
   $effective_runners = $runners.map |$r| { $runner_defaults + $r }
 
-  # Rootless runtime paths derived from the uid where known: the module
-  # itself installs the daemon socket at /run/user/<uid>/docker.sock, so the
-  # path is derivation, not configuration. The one escape hatch is narrow:
-  # runner_service.environment can point the manager service's DOCKER_HOST at
-  # a socket elsewhere. Nothing else follows it — the socket_mount volume and
-  # the healthcheck probe keep using the derived path — so it is not a general
-  # "external daemon" switch.
+  # Rootless runtime paths derived from the uid where known: the module itself
+  # installs the daemon socket at /run/user/<uid>/docker.sock, so the path is
+  # derivation, not configuration. It is the single source of the manager's
+  # DOCKER_HOST, the socket_mount volume, and the healthcheck probe, so the
+  # three never diverge. A runner whose jobs must use a different, externally
+  # managed daemon sets that per-runner via `host`, which config.toml carries
+  # natively; the manager's DOCKER_HOST is not a general external-daemon switch.
   if $runner_account['uid'] =~ Integer {
     $runtime_dir = "/run/user/${runner_account['uid']}"
     $socket_path = "${runtime_dir}/docker.sock"
@@ -336,25 +339,42 @@
     "DBUS_SESSION_BUS_ADDRESS=unix:path=${runtime_dir}/bus",
   ]
 
-  if $runner_service['manage'] and $socket_path =~ Undef and $runner_service['environment'] =~ Undef {
+  # The managed runner service always connects to the derived rootless docker
+  # socket, so it needs the uid that socket derives from. DOCKER_HOST is
+  # module-owned: there is no data-side way to point the manager elsewhere,
+  # because the healthcheck and socket_mount stay fixed to the derived socket
+  # and a hand-set DOCKER_HOST would silently diverge from them.
+  if $runner_service['manage'] and $socket_path =~ Undef {
     fail(join([
-      'rootless_gitlab_runner: the privilege-dropped runner service needs DOCKER_HOST — ',
-      'set runner_account.uid or runner_service.environment',
+      'rootless_gitlab_runner: runner_service.manage requires runner_account.uid — ',
+      'the managed service connects to the rootless docker socket derived from it',
     ]))
   }
 
-  # The inner `undef => []` arm (environment unset and no derivable socket) is
-  # reached only when runner_service.manage is off: the fail-loud guard above
-  # rejects that exact combination while the service is managed. So [] is the
-  # never-consumed value that keeps the selector total — service.pp renders the
-  # drop-in only under manage — not a deliberately empty Environment.
-  $real_service_environment = $runner_service['environment'] ? {
-    undef   => $socket_path ? {
-      undef   => [],
-      default => ["DOCKER_HOST=unix://${socket_path}"],
-    },
+  # DOCKER_HOST is derived, not configured: reject it in the environment
+  # passthrough so a hand-set value cannot diverge from the derived socket the
+  # healthcheck and socket_mount use. runner_service.environment is for any
+  # other Environment= lines the manager needs.
+  $service_environment = $runner_service['environment'] ? {
+    undef   => [],
     default => $runner_service['environment'],
   }
+  if $runner_service['manage'] and $service_environment.any |$line| { $line =~ /\ADOCKER_HOST=/ } {
+    fail(join([
+      'rootless_gitlab_runner: do not set DOCKER_HOST in runner_service.environment — ',
+      'it is derived from runner_account.uid; a runner needing another daemon uses host',
+    ]))
+  }
+
+  # The manager's Environment= lines: the derived DOCKER_HOST the executor uses
+  # to reach the rootless daemon, followed by the passthrough vars. When the
+  # service is unmanaged $socket_path may be undef; the guard above rejects that
+  # under manage, so the undef arm is the never-rendered value that keeps the
+  # selector total (service.pp renders the drop-in only under manage).
+  $real_service_environment = $socket_path ? {
+    undef   => $service_environment,
+    default => ["DOCKER_HOST=unix://${socket_path}"] + $service_environment,
+  }
 
   # The runner manager's systemd unit name is the one the gitlab-runner package
   # defines, not configuration. Derived once and shared by the service class

spec/classes/rootless_gitlab_runner_spec.rb

@@ -349,11 +349,12 @@ def account_with_uid(uid, extra = {})
     end
 
     it 'unions array subkeys across layers and removes an element via the knockout prefix' do
-      # environment is set in both layers: the common layer's DOCKER_HOST line
-      # survives the merge (arrays union under deep merge), while the node
-      # layer's '--DEBUG=1' knocks the common DEBUG=1 element out.
+      # environment is set in both layers: the common TRACE=1 survives the union,
+      # while the node layer's '--DEBUG=1' removes the common DEBUG=1. DOCKER_HOST
+      # is module-owned and renders from the merged uid (4242) alongside it.
       is_expected.to contain_file('/etc/systemd/system/gitlab-runner.service.d/10-rootless.conf')
-        .with_content(%r{^Environment=DOCKER_HOST=unix:///run/user/2000/docker\.sock$})
+        .with_content(%r{^Environment=DOCKER_HOST=unix:///run/user/4242/docker\.sock$})
+        .with_content(%r{^Environment=TRACE=1$})
         .without_content(%r{DEBUG=1})
     end
 
@@ -364,6 +365,23 @@ def account_with_uid(uid, extra = {})
     end
   end
 
+  # #26: a fleet layer sets an environment line, the node layer removes it with
+  # the '--' knockout prefix, and the deep merge yields an empty array. Because
+  # DOCKER_HOST is module-owned, the derived line renders regardless, so an
+  # emptied environment can no longer leave the managed service without a socket.
+  context 'an environment emptied through the knockout prefix' do
+    let(:node) { 'empty-env.example.org' }
+    let(:hiera_config) { File.expand_path(File.join(__dir__, '..', 'fixtures', 'hiera', 'empty_environment.yaml')) }
+
+    it { is_expected.to compile.with_all_deps }
+
+    it 'still renders the derived DOCKER_HOST when environment empties to []' do
+      is_expected.to contain_file('/etc/systemd/system/gitlab-runner.service.d/10-rootless.conf')
+        .with_content(%r{^Environment=DOCKER_HOST=unix:///run/user/2000/docker\.sock$})
+        .without_content(%r{DEBUG=1})
+    end
+  end
+
   context 'with packages listed' do
     let(:params) { { 'packages' => struct_param('packages', 'install' => %w[uidmap dbus-user-session]) } }
 
@@ -403,10 +421,10 @@ def account_with_uid(uid, extra = {})
       it { is_expected.to compile.and_raise_error(%r{set runner_account\.uid}) }
     end
 
-    context 'with the runner service and no derivable DOCKER_HOST' do
+    context 'with the runner service managed' do
       let(:params) { { 'runner_service' => struct_param('runner_service', 'manage' => true) } }
 
-      it { is_expected.to compile.and_raise_error(%r{needs DOCKER_HOST}) }
+      it { is_expected.to compile.and_raise_error(%r{runner_service\.manage requires runner_account\.uid}) }
     end
 
     context 'with a secret store present and an unresolvable token_key' do
@@ -697,6 +715,37 @@ def account_with_uid(uid, extra = {})
     end
   end
 
+  # runner_service.environment is a passthrough for additional Environment=
+  # lines; DOCKER_HOST is module-owned and rendered from the uid, so it renders
+  # alongside the passthrough vars rather than being supplied through them.
+  context 'with runner_service.environment passthrough vars' do
+    let(:params) do
+      { 'runner_service' => struct_param('runner_service', 'manage' => true,
+                                                           'environment' => ['DEBUG=1', 'BUILDX_NO_DEFAULT_ATTESTATIONS=1']),
+        'runner_account' => account_with_uid(4242) }
+    end
+
+    it 'renders the derived DOCKER_HOST alongside the passthrough vars' do
+      is_expected.to contain_file('/etc/systemd/system/gitlab-runner.service.d/10-rootless.conf')
+        .with_content(%r{^Environment=DOCKER_HOST=unix:///run/user/4242/docker\.sock$})
+        .with_content(%r{^Environment=DEBUG=1$})
+        .with_content(%r{^Environment=BUILDX_NO_DEFAULT_ATTESTATIONS=1$})
+    end
+  end
+
+  # DOCKER_HOST is derived, not configured: a DOCKER_HOST line in the passthrough
+  # would diverge from the socket the healthcheck and socket_mount use, so it
+  # fails the compile rather than rendering a manager pointed elsewhere.
+  context 'with a DOCKER_HOST line in runner_service.environment' do
+    let(:params) do
+      { 'runner_service' => struct_param('runner_service', 'manage' => true,
+                                                           'environment' => ['DOCKER_HOST=unix:///run/user/9/docker.sock']),
+        'runner_account' => account_with_uid(4242) }
+    end
+
+    it { is_expected.to compile.and_raise_error(%r{do not set DOCKER_HOST in runner_service\.environment}) }
+  end
+
   # The ownership-derivation contract (the 1.x latent wrong-owner bug): a
   # non-default account name and home must flow into every derived resource —
   # file ownership, the drop-in path, the service ExecStart, the socket path.
@@ -1047,7 +1096,7 @@ def account_with_uid(uid, extra = {})
         {
           'runner_service' => struct_param('runner_service',
                                            'manage'      => true,
-                                           'environment' => ["DOCKER_HOST=unix:///run/x\nExecStartPre=/bin/evil"]),
+                                           'environment' => ["DEBUG=1\nExecStartPre=/bin/evil"]),
           'runner_account' => account_with_uid(2000),
         }
       end