feat: share CDN bandwidth rail across copies via withCDN group id#847
Draft
juliangruber wants to merge 2 commits into
Draft
feat: share CDN bandwidth rail across copies via withCDN group id#847juliangruber wants to merge 2 commits into
juliangruber wants to merge 2 commits into
Conversation
Multi-copy CDN uploads bought a CDN bandwidth rail per copy. Thread a stable CDN group id into the withCDN metadata value so all copies of an upload resolve to one shared FWSS bandwidth rail keyed by keccak256(payer, group). The group defaults to the payer address (stable across re-uploads, preserves exact-metadata data-set reuse) and is overridable via a new cdnGroup option on Synapse and the storage/upload options. Stop forcing the withCDN value empty in synapse-core so the group id reaches the contract.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
synapse-dev | 1a9447e | Commit Preview URL Branch Preview URL |
Jun 15 2026, 05:25 PM |
Defaulting the withCDN metadata value to the payer address changed the value on every CDN data set, which broke exact-metadata data-set reuse and made StorageContext create new data sets in tests. The cdnGroup is now opt-in: empty by default (legacy one-rail-per-data-set behavior), and callers set it to share one bandwidth rail across copies.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Threads a stable CDN group id into the
withCDNmetadata value so that all copies of a multi-copy upload resolve to one shared CDN bandwidth rail in FWSS, instead of buying CDN once per copy.Full proposal, with rationale and the companion changes in the other repos: https://gist.github.com/juliangruber/a34b225f9588ec68d069054d731e20c9
The gist also links to the companion draft PRs in
FilOzone/filecoin-services,filbeam/contracts, andfilbeam/worker. The FWSS change is what gives the value meaning, it keys a shared bandwidth rail bykeccak256(payer, withCDN value).Why
Multi-copy upload stores a piece in 2 data sets on 2 providers. With
withCDNon both, the client previously bought the CDN bandwidth service twice. ThewithCDNvalue was always empty and only the key presence mattered, so there was no way to express "these data sets share one CDN subscription."How
combineMetadataaccepts acdnGroupand uses it as thewithCDNvalue, defaulting to an empty string.synapse-coredatasetMetadataObjectToEntryno longer forces thewithCDNvalue to empty when the cdn flag is set, it preserves an existing value (the group id) and only defaults to empty when the key is absent. Without this the value never reached the contract.cdnGroupoption onSynapseand the storage/upload options.Reviewer notes
The default group value moves from empty to the payer address. Existing CDN data sets created with an empty
withCDNvalue will no longer match the new default undermetadataMatches, so the next CDN upload creates a fresh data set rather than reusing an old one. A caller can passcdnGroup: ''to match pre-existing empty-value data sets. An address is 42 chars, well within the metadata value limit, customcdnGroupvalues must respect it.Follow-ups
These are intentionally out of scope for this draft.