Skip to content

feat: write cache misses with a single setMany - #21

Merged
mvantellingen merged 3 commits into
mainfrom
feat/batch-cache-writes
Aug 19, 2026
Merged

feat: write cache misses with a single setMany#21
mvantellingen merged 3 commits into
mainfrom
feat/batch-cache-writes

Conversation

@mvantellingen

Copy link
Copy Markdown
Member

toCache issued a set per cache key and never collected the promises:

for (const [key, value] of items) {
	options.store.set(key, value, options.ttl);
}

Found while looking at a category page trace in a storefront that uses this
through @evolve-framework/commercetools' dataloaders: 35 redis-SET spans in
one request
, against 6 redis-MGET — reads batch through store.get(keys),
writes did not. Over two hours redis-SET was the single most-emitted span in
that service (1021 spans, P50 99ms, P95 496ms), which drowned out everything else
in the trace.

What this is not

Not a round-trip win. Measured against a real Redis by patching
net.Socket.prototype.write, 35 keys of ~2KB, three runs:

path socket writes all writes dispatched within complete
for (…) store.set(), not awaited (before) 35 0.2 ms 3.4–3.9 ms
for (…) await store.set() (hypothetical) 35 27 ms 27 ms
store.setMany() (this PR) 36 (MULTI + 35× SET) 0.1 ms 1.6–1.9 ms

The old writes went out within 0.2 ms without waiting for replies, so the client
already pipelined them. Same command count, same bytes (76690 vs 76675).

What it is

  • One batch instead of one command per key. @keyv/redis.setMany builds a
    single MULTI/EXEC, so tracing reports one batch span instead of 35.
  • The writes are awaited, so a failing store surfaces as an error event on
    it rather than as a floating rejection.

The keyv floor

setMany needs keyv >= 5.3.4; the peer range moves from >=4.0.0 <6.0.0.
Before 5.3.4 Keyv.setMany hands entries straight to the store adapter with no
{value, expires} wrapper, no serialization and no key prefix — so it writes
values get cannot read back, silently. Verified per version:

5.3.3: raw entries    5.3.4: serializes    5.4.0: serializes    5.5.x: serializes

Verification

pnpm test (8 passed) and pnpm check green, plus against a real Redis with
MONITOR attached:

  • 8 keys through DataLoaderCache produce one MULTI … 8× SETEXEC
  • a second loader that throws if it reaches the origin serves all 8 from Redis,
    so setMany writes what get parses
  • set and setMany land on byte-identical keys, both with a TTL — otherwise a
    release would silently invalidate every cached entry

@changeset-bot

changeset-bot Bot commented Aug 19, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 97f2c75

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@labdigital/dataloader-cache-wrapper Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

toCache issued a set per cache key and never collected the promises. The
individual writes were already pipelined by the store's client, so this is not a
round-trip win -- the same commands and bytes go over the socket either way --
but the batch is now one call into the store, which the redis adapter wraps in a
single MULTI, and instrumentation reports one batch instead of one command per
key. A category page loading 36 products emitted 36 redis-SET spans per request.

The writes are also awaited, so a failing store surfaces as an error event on it
rather than as a floating rejection.

setMany needs keyv 5.3.4 or newer. Before that, keyv hands unserialized entries
straight to the store adapter, without the value wrapper or the key prefix that
get applies on read, so the writes land as values get cannot read back.
@mvantellingen
mvantellingen force-pushed the feat/batch-cache-writes branch from 82ce2af to c5857c0 Compare August 19, 2026 17:34
The tsdown build failed on the 20.x job. tsdown only uses its native config
loader when `import-without-cache` is supported (node ^22.18.0 || >=24.0.0);
below that it falls back to the `unrun` loader, which is an optional peer
dependency and not installed here. Node 20 is end-of-life anyway.

engines.node moves from >=20 to >=22 to match.
@mvantellingen
mvantellingen merged commit 5b0d7a5 into main Aug 19, 2026
5 checks passed
@mvantellingen
mvantellingen deleted the feat/batch-cache-writes branch August 19, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant