ENG-525: provider-aware warm pool

[czpython]

What

Makes the warm-host pool provider-aware. The pool was single-dimensioned (one POOL_SIZE, all hosts warmed with the default provider), so a warm host for one provider could never satisfy a request pinned to another.

• Config: new POOL_SIZES JSON env ({"exe": 2, "hetzner": 1}) is the primary sizing knob. Effective targets compose in Settings.get_pool_targets(): POOL_SIZE seeds the default provider, POOL_SIZES entries override it, zero targets drop out. Values are validated >= 0. POOL_MAX_CREATES_PER_TICK stays a single global blast cap; POOL_HOST_MAX_AGE_HOURS unchanged.
• Maintenance (hosts.pool): counts warm members per provider in one GROUP BY query, then tops up / sheds each provider toward its own target. The per-tick create budget is round-robined across providers so one large deficit can't starve the others. The excess sweep covers the union of configured and present providers, so a provider dropped from the targets sheds to zero instead of idling until its max-age TTL. create_host targets the specific provider.
• Claim path (get_or_create_host): a pool-eligible request (default image, no env) resolves its provider (provider or DEFAULT_HOST_PROVIDER) and claims only warm hosts WHERE provider == P, gated on P having a warm target. Pinned providers now claim from their own pool instead of always provisioning fresh; a miss falls through to fresh provisioning against P with the same response shape.

POOL_SIZE alias decision

Kept POOL_SIZE working as an alias for the default provider's target (the ticket's stated default, per no-silent-removals): existing deployments keep their warm pool without a config migration. Precedence is explicit — a POOL_SIZES entry for the same provider wins, and an explicit {"exe": 0} disables that provider's pool even when the alias would seed it.

Acceptance criteria

• With targets configured per provider, each provider maintains its own warm count independently
• A POST /hosts for provider P (default image, no env) claims a warm P host and never a host of another provider
• Over-provision from overlapping ticks is shed per-provider on the next tick (existing excess path, now provider-scoped)
• POOL_MAX_CREATES_PER_TICK still bounds total creates across all providers per tick

Codex review

Round 1 flagged one P2: the round-robin batch was built from a sorted provider order, so a per-tick cap smaller than the number of underfilled providers always spent its budget on the same leading provider — permanently starving the rest if the leader's creates kept failing. Fixed by shuffling the underfilled-provider order each tick (there is no cross-tick state to rotate a cursor through); the within-tick round-robin is unchanged, and a regression test pins that a permanently failing provider cannot starve the others.

Verification

uv run ruff check, uv run ruff format --check, uv run pyright, uv run pytest (282 passed). New tests cover target composition/validation, per-provider top-up and shed, dropped-provider shed, cross-provider cap with round-robin, provider-scoped claim, the no-cross-provider-claim guarantee, and no-starvation under a tight cap with a failing provider.

🤖 Generated with Claude Code