fix(ci): stop three timing races failing unrelated PRs - #213
Conversation
|
Warning Review limit reached
Next review available in: 8 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
commit: |
Three tests were failing intermittently and blocking every PR on the required Test check. Each raced something instead of waiting for it. **The deploy-lease heartbeat** forked a 10ms heartbeat, slept 100ms of real time, then asserted at least two had fired — so a starved event loop failed it (`Expected: >= 2`). It now waits until the second heartbeat is observed, which cannot fail that way at any speed. I could not reproduce the failure locally even under heavy load, so the case for this rests on the shape of the test rather than on a local repro. **The emulator daemon restart** failed with "A Prisma Dev server with the name ... is already running", not a timeout. `terminate` gave SIGTERM 5s before SIGKILL, and a SIGKILLed daemon never closes its in-process `@prisma/dev` servers, so their name locks leak and the next daemon has to wait out the stale threshold. It also returned immediately after SIGKILL, letting a replacement start while the old process still held its ports. SIGKILL now waits for the process to be reaped, and the grace period is 20s — spent only on a daemon that is genuinely not exiting, since the wait ends as soon as the pid is gone. All 11 emulator tests still finish in 35s together, so a clean stop is unaffected. **The database lifecycle test** was the one real timeout: 30151ms against a ~8s normal run. Raised to 60s, matching the 45-120s its heavier neighbours already use. No race to fix here — every step is awaited, it is simply slow. Also makes `deleteProjectDeep` report a thrown transport error as "not gone" rather than propagating it. The sweep in ci-cleanup.ts deletes projects in a loop, so one socket error abandoned every project after it; the post-teardown retry now also rides out a blip. Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
cb419e0 to
a0b7598
Compare
Three tests have been failing intermittently and blocking every PR on the required
Testcheck, including #211 (which needed four attempts) and #212. None of them relate to the changes they were failing. Each races something instead of waiting for it.The deploy-lease heartbeat
state-api.test.tsforked a heartbeat on a 10ms interval, slept 100ms of real time, then asserted at least two had fired:A starved event loop fails that, which is what CI saw (
Expected: >= 2). It now waits until the second heartbeat is observed, so it cannot fail that way at any speed — and it finishes sooner than the fixed sleep on a fast machine.Honest caveat: I could not reproduce the original failure locally, 20 runs under eight CPU-burning processes. The case for this change rests on the shape of the test, not on a local repro.
The emulator daemon restart
This one was misdiagnosed as a timeout in earlier discussion. It has a 45s budget and failed at 27.5s with:
The daemon hosts its
@prisma/devservers in-process, and only a graceful exit closes them and releases each server's name lock.terminategave SIGTERM 5 seconds before SIGKILL, so a daemon slow to shut down under load got killed with its locks still held. The next daemon then had to wait out proper-lockfile's stale threshold, which exhaustedpostgres-main.ts's already-generous "already running" retry budget (2 retries × 12s of polling).It also returned immediately after SIGKILL, which is a bug in its own right: SIGKILL is delivered asynchronously, so
stopDaemoncould report success while the old daemon still held its ports and locks.Both are fixed. SIGKILL now waits for the process to actually be reaped, and the grace period is 20s. That budget is only ever spent on a daemon that is genuinely not exiting, because the wait ends as soon as the pid is gone — all 11 emulator tests still complete in 35s together, which they could not if teardown were paying it.
The database lifecycle test
The only genuine timeout: 30151ms against a ~8s normal run. Raised to 60s, matching the 45-120s its heavier neighbours in the same file already use. There is no race to fix here — every step is awaited and it is simply slow — so the timeout is the right mechanism and it was set too tight. The idempotency test beside it has identical setup cost and gets the same budget.
Also: one transport error no longer abandons a whole sweep
deleteProjectDeepnow reports a thrown transport error as "not gone" instead of propagating it.ci-cleanup.tssweeps projects in a loop, so a single socket error abandoned every project after it, leaking workspace slots until the next run. The post-teardown project-delete retry now also rides out a blip rather than giving up.Two tests cover it, and both fail without the change.
Verification
🤖 Generated with Claude Code