Proposed publisher API, generated from the consumption specification - #147
Closed
0x73746F66 wants to merge 437 commits into
Closed
Proposed publisher API, generated from the consumption specification#1470x73746F66 wants to merge 437 commits into
0x73746F66 wants to merge 437 commits into
Conversation
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Pavel Shukhman <taleodor@users.noreply.github.com>
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Pavel Shukhman <taleodor@users.noreply.github.com>
Signed-off-by: Piotr P. Karwasz <piotr@github.copernik.eu>
Signed-off-by: Piotr P. Karwasz <piotr@github.copernik.eu>
Signed-off-by: Piotr P. Karwasz <piotr@github.copernik.eu>
Signed-off-by: Piotr P. Karwasz <piotr@github.copernik.eu>
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Olle E. Johansson <oej@edvina.net>
Signed-off-by: Olle E. Johansson <oej@edvina.net>
Signed-off-by: Olle E. Johansson <oej@edvina.net>
Classifier discussion moved to CycloneDX#131 Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
feat: add release into spec, further clean up
Signed-off-by: Piotr P. Karwasz <piotr@github.copernik.eu>
Clarify TEA Product fields
`examples` must be an array, not an object Signed-off-by: Piotr P. Karwasz <piotr@github.copernik.eu>
fix: Examples syntax
Signed-off-by: Piotr P. Karwasz <piotr@github.copernik.eu>
Clarify TEA Component fields
Signed-off-by: Olle E. Johansson <oej@edvina.net>
Signed-off-by: Olle E. Johansson <oej@edvina.net>
Signed-off-by: Olle E. Johansson <oej@edvina.net>
Signed-off-by: Olle E. Johansson <oej@edvina.net>
Add `version` field to TEA Artifact
…iscussion) Signed-off-by: Paul Horton <phorton@sonatype.com>
Signed-off-by: Paul Horton <phorton@sonatype.com>
`reuslts` is now required in Products Response Signed-off-by: Paul Horton <phorton@sonatype.com>
Signed-off-by: Paul Horton <phorton@sonatype.com>
Signed-off-by: Paul Horton <phorton@sonatype.com>
Signed-off-by: DR <eaglei15@gmail.com>
Update tea-implementations.md adding CyBeats
Adding CyBeats implementation
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
Signed-off-by: Pavel Shukhman <pavel@reliza.io>
…gination feat: Add opaque pagination
…-inline-object-class-names fix: correct class naming on generation following pagination
…-documentation docs: TEA UUID Scope and Stability (CycloneDX#217)
Contributor
Author
|
Update PR description, and rebased main. Publisher spec is not generated, so consumer updates are shared with publisher spec going forward |
…s and distribution
The document did not validate as OpenAPI 3.1, and the way it failed
mattered. `components.operations` is not a field OpenAPI defines, so the
three $refs pointing into it never resolved and every delete operation
declared no responses at all — a generator reading this produced deletes
that returned nothing. Redocly reported 9 errors; it now reports 0.
Error responses carried `application/json: {}`, an empty schema that says
nothing about what comes back. Publication runs in release pipelines, so
failures need to be actionable without a human reading prose: there is
now an `error` schema with a machine-readable code.
Three things were missing rather than broken.
Artifacts. The draft could create products, components, releases and
collections but had no way to publish the artifact that carries the SBOM,
VEX or attestation the exchange exists to move. Creating an artifact and
uploading its bytes are separate operations, because an SBOM is routinely
tens of megabytes and a publisher who must resend all of it to fix one
metadata field will not fix the field. Content goes up per format in its
own media type rather than base64 in JSON, stored bytes are immutable so
a recorded checksum cannot start describing different content, and
Content-Digest is verified so a truncated retry fails loudly instead of
publishing corruption under a checksum that claims otherwise.
Signatures. The consumption API exposes only a signatureUrl, leaving a
consumer to infer the scheme from the bytes. The publisher knows it, so
artifactSignature records the scheme, key id, certificate chain and
transparency-log entry — which is what makes a signature checkable rather
than merely present.
Distribution. TEA describes how data is fetched, not who may fetch it.
That boundary works for a consumption API and cannot hold for a
publication one, where the same server holds public material, material
shared with named counterparties, and internal material. Three
visibilities, plus mirroring to other TEA servers. Inheritance narrows
and never widens — a child may restrict what it inherits and a server
rejects any attempt to loosen it, because otherwise marking a product
private would guarantee nothing when any artifact beneath it could be
made public. GET /distribution/{uuid} reports declared against effective
and names the ancestor responsible, since the gap between those two is
where accidental disclosure hides.
Idempotency-Key on creates, because pipelines retry and a timeout that
actually succeeded otherwise yields a duplicate found by a consumer
rather than by the publisher.
The README records what changed and three open questions: releases are
not split into product and component as the consumption API splits them,
there is no publisher-side read surface, and publishTo does not say
whether a failed mirror blocks local publication.
Signed-off-by: Christopher Langton <chris@vulnetix.com>
# Conflicts: # contributors.md # spec/publisher/openapi.json
…d the drift
The publisher document was a hand-maintained copy of the consumption
specification and had fallen four minor versions behind it. The
consumption API is at 0.4.0; the copy still carried schemas from 0.0.3,
renamed along the way — artifactFormat beside the real artifact-format,
typeUuid beside uuid, a private artifactChecksum beside checksum. A
combined document therefore held two definitions of the same concept,
each free to move independently, with nothing to say which one a
publisher should believe.
It is now generated. build.mjs merges spec/openapi.yaml with a
publication overlay, so everything shared is a $ref into the consumption
specification's own definitions and the publisher version is copied from
the consumer rather than chosen. The build refuses to run if the overlay
declares a schema, parameter, response, request body or path method the
consumption specification already owns — the overlay may only add.
Alignment corrections this surfaced:
- Releases are split into /productRelease and /componentRelease. The
consumption API has distinguished the two since 0.1.0 and a single
/release could not say which it was creating.
- Collections are addressed through their release. The consumption
specification states a collection's UUID matches its release and that
an update only changes the version, so it is not an independently
created object: PUT /productRelease/{uuid}/collection, not
POST /collection.
- Errors reuse the consumption envelope. publisher-error-response has the
same { error } shape as error-response over an enumeration that repeats
OBJECT_UNKNOWN and OBJECT_NOT_SHAREABLE and adds the failures only a
writer can hit, so clients branch on one enumeration rather than two.
- distribution became access-policy, because the consumption
specification already uses release-distribution for a downloadable
build with its own URL and checksums. Reusing the word for access
control would have been actively misleading.
Enforcement of the access policy needs no new consumer behaviour: a
denied consumer gets OBJECT_NOT_SHAREABLE, which already exists in
unknown-error-type.
The generated document now adds no lint errors of its own. It reports 24,
and linting spec/openapi.yaml alone reports the same 24 — 23 operations
missing summaries and one OpenAPI 3.0 `nullable` on
pagination-details.nextPageToken. Those belong upstream rather than
patched here, where they would put the fork at odds with a file upstream
owns. CI compares the two counts and fails only if the overlay makes
things worse.
Signed-off-by: Christopher Langton <chris@vulnetix.com>
createTeaDistribution and deleteTeaDistribution, on the component release because that is where the consumption schema puts release.distributions. distributionId is absent from the request body by design: the server derives it, and a publisher-supplied identifier would let one tenant claim another's. identifiers stays optional — Homebrew and Scoop have no accepted purl type, and an identifier naming a coordinate that does not resolve is worse than none, because a consumer will follow it. Regenerated openapi.yaml. Redocly reports the same 24 errors as before, all inherited from the upstream consumption specification. Signed-off-by: Christopher Langton <chris@vulnetix.com>
Audited the overlay against a working TEA server and its console — Vulnetix
Resolve — rather than against itself. Five things the specification could not
express, each of which the implementation had to answer on its own.
A product release could not state its components. The consumption API lists
`components` among productRelease's required members, so every release a
conformant publisher could create had to be served with an empty list: a claim
about the product rather than a gap in the record.
An artifact could not say which distributions it describes. `distributionIds`
exists in the consumption artifact and was unreachable from the publication
side, which matters exactly where an SBOM is not one document.
There was no publisher read surface, and the consumption API cannot substitute
for one: it answers what a reader is entitled to see, so an object correctly
withheld and an object that was never created are indistinguishable from
outside. `GET /publications` and `/publications/{uuid}/releases` report what
was published alongside the policy in force for it.
`public` asked for "a separate confirmation" without naming one, so the
implementation invented `?confirm=public`. An unspecified requirement is one
every publisher satisfies differently, which is the failure this document
exists to prevent, so the parameter is now specified.
`publishTo` named targets nothing could configure and left failure semantics
undefined. Targets are now registered through `/publicationTargets`, mirroring
is asynchronous and never blocks the local write, per-target state is reported
on the access policy, and a server that does not mirror answers
`MIRRORING_UNSUPPORTED` instead of accepting the request and discarding it.
One change to the consumption specification: an empty alternative in the global
`security`. Without it `visibility: public` cannot mean what it says, because
discovery has no step at which a credential could be obtained and a server that
answers an anonymous request is non-conformant for doing so. It is a widening,
so nothing conformant breaks. build.mjs gives every overlay operation an
explicit credential requirement in return, so anonymous writes stay
non-conformant.
Redocly still reports the same 24 errors as the consumption specification
alone; the overlay adds none.
Signed-off-by: Christopher Langton <chris@vulnetix.com>
0x73746F66
changed the title
0x73746F66
marked this pull request as ready for review
August 2, 2026 10:43
The step has failed on every run since it was added, and not for the reason it looks like: `npx redocly lint` exits non-zero whenever it reports an error, both documents report 24 by inheritance, and GitHub Actions runs the step under `bash -e`. With `set -o pipefail` the pipeline inherits that exit status, so errexit killed the script at the first assignment, before either count was echoed or compared. The log showed an exit code and no output, which is what made it read as a lint regression. Discard redocly's exit status in the helper, where the count already carries everything the status would have told us. Two things worth keeping while in here. Empty output now fails loudly rather than counting as zero errors, because a gate that passes when the linter did not run is worse than no gate. And the diagnostic lint in the failure branch gets `|| true`, so errexit cannot cut it short before the explicit exit. Verified against the step body extracted from the workflow itself, under `bash -e`: equal counts exit 0; a linter producing no output exits 1 with a message; a publisher document carrying one extra error exits 1 and reports the delta. Signed-off-by: Christopher Langton <chris@vulnetix.com>
The inheritance rule said a child may narrow what it inherits and must not widen it, and described resolution as taking the nearest declared policy. Those two statements do not combine into the guarantee the document claims. ACCESS_WIDENS_PARENT rejects a declaration that is too wide at the moment it is written. It says nothing about a parent narrowed afterwards. So under nearest-wins, an artifact that had declared public goes on being public once its product is set to private, because the artifact's own declaration is the nearest one and therefore becomes the answer. Marking a product private is then a statement about a single row, which is the opposite of what the section says it is for. Resolution is now the narrowest declaration anywhere between the object and the root. Where several levels declare shared, entitlements intersect and the earliest expiry applies, since a grant on a child cannot create an entitlement its parent withheld. Descendants' declarations are explicitly not rewritten when an ancestor narrows. Folding is reversible and rewriting is not: widening a product back restores each descendant to its own declaration and no further, which is what makes a temporary embargo something a publisher can undo. Redocly still reports the same 24 errors as the consumption specification alone. Signed-off-by: Christopher Langton <chris@vulnetix.com>
Every em dash in the publisher overlay, its README, the generator and the one comment added to the consumption specification is rewritten. Each was replaced according to the grammar of its own sentence, using a colon, a comma, a full stop or a parenthesis as the clause required, rather than by substituting one character for another. Wording only. `npm run check` reports the document still in sync, and redocly reports the same 24 errors on both documents. Signed-off-by: Christopher Langton <chris@vulnetix.com>
Contributor
Author
|
@oej i've added open questions in the PR description, but so far as having a consumer compliant publisher spec, generating and extending fromt he consumer is hard to argue agianst, if there is no objections we should merge this and break out the outstnadings into discrete work |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposes a publisher API for TEA: how software gets into a transparency log. The
consumption specification already says how it comes out.
spec/publisher/openapi.yamlis generated. It isspec/openapi.yamlplus thepublication operations in
spec/publisher/overlay.yaml, merged byspec/publisher/build.mjs. Anything the two share is a$refinto theconsumption definitions, and the build fails if the overlay redefines something
the consumption specification already owns. The publisher version is the
consumer version by construction.
The earlier draft was a hand-maintained copy and had drifted to 0.0.3 schemas
under parallel names:
artifactFormatbesideartifact-format,typeUuidbeside
uuid. Generating the document removes that possibility instead ofcorrecting the symptom. Reasoning for each decision is in
spec/publisher/README.md.What the overlay adds
productReleaseandcomponentReleaseto match the consumption model.PUT /productRelease/{uuid}/collection, because the consumption specification saysa collection's UUID is its release's and an update only bumps the version.
updateReasonis required.often tens of megabytes, and a publisher who has to resend all of it to fix
one metadata field will not fix the field. Uploads verify
Content-Digest(RFC 9530).
transparency-log entry. The consumption API exposes only
signatureUrl, so aconsumer currently has to work out the scheme from the bytes before it can
verify anything.
known about it. They hang off a component release, which is where
release.distributionsis defined.private,shared(named organisations, optional expiry)or
public, inherited from artifact to collection to release to product.GET /publications, reporting what an organisation has published and who canread it.
publishTo, mirroring to targets registered at/publicationTargets.Idempotency-Keyon creates, because pipelines retry.Two rules in the access policy do most of the work. Inheritance narrows and
never widens, so a server refuses an attempt to loosen a child with
ACCESS_WIDENS_PARENT; without that, marking a product private would guaranteenothing, because any artifact beneath it could be made public on its own. And a
change to
publicrequiresconfirm=public, because no later request recallswhat has already been fetched.
None of it asks anything new of a consumer. A reader denied by a policy gets the
existing
OBJECT_NOT_SHAREABLE.One change to the consumption specification
spec/openapi.yamlgains an empty alternative in its globalsecurity, makingauthentication optional. This is the only edit to a file the working group owns,
and the only part of this PR that needs a decision rather than a review.
Without it the publication API cannot mean what it says.
visibility: publicisdefined as readable without authentication, and the discovery sequence (take the
domain from a TEI, fetch that host's discovery document, call the root it names)
has no step at which a credential could be obtained. A specification that
requires one everywhere closes its own entry point, and a server that does serve
a public object anonymously is non-conformant for it.
The change only widens what is allowed, so nothing conformant breaks. A server
may still refuse every anonymous request. Every operation in the consumption
document is a read, so nothing becomes writable. In exchange,
build.mjsgiveseach of the 24 write operations an explicit
bearerAuth/basicAuthrequirement, so anonymous writes stay non-conformant.
If the working group would rather leave the document alone, the alternative is
prose saying servers may relax it for public objects, and accepting that the
document and the behaviour disagree.
Compatibility with 0.4.0
The consumption specification is otherwise untouched.
diffbetween this branchand the copy that implementation validates against reports the security block
above and nothing else.
redocly lintreports 24 errors on the generated document and the same 24 onspec/openapi.yamlalone: 23 missing operation summaries and onenullableinpagination-details. CI compares the two counts and fails if the publisherdocument is ever worse than the specification it is built from.
Checked against a deployed implementation, Vulnetix Resolve.
23 of 23 consumption operations exercised, 160 test cases, 0 failures, 85
responses validated against the upstream schemas and all conforming. The
Insights API adds 2 operations and 27 cases, with 16 responses validated against
CycloneDX 1.6.
All four access modes work end to end. An anonymous consumer resolves a TEI at
the root advertised in
/.well-known/tea, then reads a public release, itscollection, its artifacts and their bytes with no credential at any step.
sharedgrants name organisation UUIDs with an optional expiry, and an expiredgrant is not a grant.
privateis the default and owner-only, withGET /publicationsas the way an owner confirms it. Discovery resolves bothurn:tei:uuid:andurn:tei:purl:, listings filter onidTypeandidValue,and retrieval walks release to collection to artifact to content. A refusal
answers
OBJECT_UNKNOWNand not 403, so nobody can enumerate anotherpublisher's UUIDs.
Those 160 cases cover the consumption operations. The publication operations are
implemented and in use, and the implementation now checks itself against this
document: every error code it emits must be a value
publisher-error-typedeclares, and every operation this document declares must be routed. They are
not yet driven by a conformance suite. That is the next piece of work, not a
claim being made here.
What implementing it found
Five things this document could not express, all fixed in it now.
A product release could not state its
components, which the consumptionproductReleaselists as required. An artifact could not setdistributionIds.publicasked for a separate confirmation without saying what one was, so ourimplementation uses
?confirm=public. AndpublishTonamed targets thatnothing could configure, with failure semantics the README itself admitted were
undefined.
Open questions
Artifact versions. The consumer API addresses artifacts as
/artifact/{uuid}/{artifactVersion}, but create and update do not say how apublisher advances that version.
Mirror authentication runs one way only. Nothing says how a receiving server
decides whether to accept a mirrored object, or how it records that the object
arrived as a copy. Consensus across transparency logs requires this to be resolved, or consensus is not possible and there is nothing trustworthy across transparency exchange servers.