Skip to content

[2.x] feat: pausable queues#4841

Merged
imorland merged 1 commit into
2.xfrom
im/pausable-queues
Jul 26, 2026
Merged

[2.x] feat: pausable queues#4841
imorland merged 1 commit into
2.xfrom
im/pausable-queues

Conversation

@imorland

Copy link
Copy Markdown
Member

Implements the queue pause/resume surface that QueueFactory has stubbed since #4673/#4683, removing the TODO. Works across all queue backends — database, redis, and Horizon workers all consult this same factory when popping jobs.

How it works

Illuminate's Worker filters paused queues in getNextJob() via the manager's pause surface. QueueFactory now answers for real, backed by the shared cache store with Illuminate's exact key format (illuminate:queue:paused:{connection}:{queue}) — so the signal reaches every worker process, across containers, and interoperates with anything speaking the upstream convention. QueuePaused/QueueResumed events are dispatched for parity.

  • Per-queue + wildcard: pause($conn, 'media') or pause($conn, '*') — the wildcard covers every queue on the connection, so pause-all needs no queue enumeration (core can't portably enumerate queue names across backends). The layers are orthogonal: resume-all doesn't clear individually paused queues.
  • pausedQueues($connection) — a Flarum addition tracking what's paused, self-healing when pauseFor() TTLs expire or pauses are cleared externally. Powers the dashboard warning.
  • queue:pause / queue:resume — Flarum-native commands (the stock Illuminate ones type-hint the concrete QueueManager). Accept queue or connection:queue, default to the active connection, support --all. Registered only for non-sync connections.
  • POST /api/queue/pause — admin-gated; refuses with 409 on the sync driver (inline execution, nothing to pause).
  • flarum.queue.queues container binding — queue names known to the installation (default ['default']); extensions routing jobs via AbstractJob::$onQueue append theirs to get per-queue admin controls.

Admin UI

  • Dashboard: non-dismissible warning whenever any queue is paused (wording distinguishes "all" from named queues), with a Manage link — same pattern as the maintenance-mode alert.
  • Advanced page: with one known queue, a single "Pause queue processing" switch (wired to the wildcard); with several, a pause-all switch plus per-queue switches. No controls on sync.

Validation

  • 17 integration tests: factory semantics (cache key format, wildcard, orthogonality, self-healing list, events), worker pop behaviour on the database driver, command registration/behaviour, endpoint auth/sync-refusal. Full suite 743 green, PHPStan clean.
  • Live-verified on a Horizon deployment: while paused, a pushed job sat in redis; after queue:resume it drained within seconds — commands issued from the web container, workers running in a separate container.

Operator note

Long-running workers evaluate pause state with the code they booted with — a one-time queue:restart after upgrading picks up the new factory.

Follow-ups (fof/horizon)

Register supervisor queue names into flarum.queue.queues; supervisor-profile extender with grouped pause controls; cross-container horizon:pause/continue.

@imorland imorland added this to the 2.0.0-rc.6 milestone Jul 26, 2026
@imorland imorland added type/feature javascript Pull requests that update Javascript code and removed javascript Pull requests that update Javascript code labels Jul 26, 2026
Illuminate's Worker consults the queue manager's pause surface when
popping jobs, but Flarum's QueueFactory stubbed the whole family as
no-ops. The stubs are now real, backed by the shared cache store using
Illuminate's own key format (illuminate:queue:paused:{connection}:{queue}),
so the signal reaches every worker process — across containers, and
including Horizon workers, which resolve this same factory.

- Per-queue pause plus a wildcard (*) that covers every queue on the
  connection, so pause-all works without core having to enumerate queue
  names. The layers are orthogonal: resuming all does not clear
  individually paused queues.
- pausedQueues() tracks what is paused per connection for the dashboard,
  self-healing when a pauseFor() TTL expires or a pause is cleared
  through the raw cache key.
- queue:pause / queue:resume commands (Flarum-native: the stock
  Illuminate ones type-hint the concrete QueueManager). The queue
  argument accepts queue or connection:queue and defaults to the active
  connection; --all pauses or resumes everything. Registered only for
  non-sync connections, like the other queue commands.
- POST /api/queue/pause admin endpoint; refuses (409) on the sync
  driver, where jobs execute inline and there is nothing to pause.
- flarum.queue.queues container binding: the queue names known to the
  installation (default: ['default']). Extensions routing jobs onto
  named queues can append theirs to get per-queue admin controls.
- Admin UI: a warning on the dashboard whenever any queue is paused,
  and pause switches on the Advanced page — a single switch when only
  one queue is known, per-queue switches plus pause-all otherwise.
  Sync driver shows no pause controls.

Note for operators: long-running workers evaluate the pause state with
the code they booted with, so a one-time queue:restart is needed after
upgrading to pick up the new factory.
@imorland
imorland force-pushed the im/pausable-queues branch from 15e158a to ba7d2d0 Compare July 26, 2026 22:17
@imorland
imorland marked this pull request as ready for review July 26, 2026 22:19
@imorland
imorland requested a review from a team as a code owner July 26, 2026 22:19
@imorland
imorland merged commit 7fe9d66 into 2.x Jul 26, 2026
25 checks passed
@imorland
imorland deleted the im/pausable-queues branch July 26, 2026 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant