Skip to content

feat: share CDN bandwidth rail across copies via withCDN group id#847

Draft
juliangruber wants to merge 2 commits into
masterfrom
cdn-shared-bandwidth-rail
Draft

feat: share CDN bandwidth rail across copies via withCDN group id#847
juliangruber wants to merge 2 commits into
masterfrom
cdn-shared-bandwidth-rail

Conversation

@juliangruber

Copy link
Copy Markdown
Member

What

Threads a stable CDN group id into the withCDN metadata value so that all copies of a multi-copy upload resolve to one shared CDN bandwidth rail in FWSS, instead of buying CDN once per copy.

Full proposal, with rationale and the companion changes in the other repos: https://gist.github.com/juliangruber/a34b225f9588ec68d069054d731e20c9

The gist also links to the companion draft PRs in FilOzone/filecoin-services, filbeam/contracts, and filbeam/worker. The FWSS change is what gives the value meaning, it keys a shared bandwidth rail by keccak256(payer, withCDN value).

Why

Multi-copy upload stores a piece in 2 data sets on 2 providers. With withCDN on both, the client previously bought the CDN bandwidth service twice. The withCDN value was always empty and only the key presence mattered, so there was no way to express "these data sets share one CDN subscription."

How

  • combineMetadata accepts a cdnGroup and uses it as the withCDN value, defaulting to an empty string.
  • synapse-core datasetMetadataObjectToEntry no longer forces the withCDN value to empty when the cdn flag is set, it preserves an existing value (the group id) and only defaults to empty when the key is absent. Without this the value never reached the contract.
  • The storage manager resolves the group once and applies it to the primary context and every secondary, including replacement/retry secondaries, so all copies land in the same subscription. The group defaults to the payer (client) address, which is stable across re-uploads and keeps exact-metadata data-set reuse intact, and is overridable with a new cdnGroup option on Synapse and the storage/upload options.

Reviewer notes

The default group value moves from empty to the payer address. Existing CDN data sets created with an empty withCDN value will no longer match the new default under metadataMatches, so the next CDN upload creates a fresh data set rather than reusing an old one. A caller can pass cdnGroup: '' to match pre-existing empty-value data sets. An address is 42 chars, well within the metadata value limit, custom cdnGroup values must respect it.

Follow-ups

These are intentionally out of scope for this draft.

  • Decide the canonical default group (payer address vs an explicit per-logical-dataset id) with the FWSS PR.

Multi-copy CDN uploads bought a CDN bandwidth rail per copy. Thread a
stable CDN group id into the withCDN metadata value so all copies of an
upload resolve to one shared FWSS bandwidth rail keyed by
keccak256(payer, group). The group defaults to the payer address (stable
across re-uploads, preserves exact-metadata data-set reuse) and is
overridable via a new cdnGroup option on Synapse and the storage/upload
options. Stop forcing the withCDN value empty in synapse-core so the
group id reaches the contract.
@github-project-automation github-project-automation Bot moved this to 📌 Triage in FOC Jun 15, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 15, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
synapse-dev 1a9447e Commit Preview URL

Branch Preview URL
Jun 15 2026, 05:25 PM

@BigLep BigLep moved this from 📌 Triage to ⌨️ In Progress in FOC Jun 15, 2026
Defaulting the withCDN metadata value to the payer address changed the
value on every CDN data set, which broke exact-metadata data-set reuse
and made StorageContext create new data sets in tests. The cdnGroup is
now opt-in: empty by default (legacy one-rail-per-data-set behavior),
and callers set it to share one bandwidth rail across copies.
@BigLep BigLep moved this from ⌨️ In Progress to 🐱 Todo in FOC Jun 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: 🐱 Todo

Development

Successfully merging this pull request may close these issues.

2 participants