July release candidate (DO NOT MERGE) - #919
Draft
ChristianPavilonis wants to merge 320 commits into
Draft
Conversation
Introduce an operator-armed render-trace overlay so the winning auction recorded in window.tsjs.renders can be confirmed visually on the page, on both the SSAT/GAM and /auction paths. Server: GET /_ts/trace toggles a host-only ts-trace cookie and redirects to /, gated by the new [debug] trace_route_enabled flag (404 when off). Registered on all four adapters (Fastly/Cloudflare/Axum/Spin) and documented in trusted-server.example.toml. Client: while the cookie is armed, TSJS draws a floating Google-Publisher-Console-style panel summarising every traced slot and a confirmation badge on each genuinely-rendered creative. The panel reports honest per-slot status — ok / hidden / gam-only / empty — derived from separate signals (gamEmpty from GAM's slotRenderEnded, injected = whether TS actually placed the creative, visible = ancestor-aware visibility) so it never overclaims a render GAM merely fired. Clicking a row copies the full record; the badge is shown only on ok slots.
The prod page runs two independent auctions against the same placements: the one-time SSAT auction on navigation, and an ongoing client-side /auction driven by GAM refresh through the Prebid.js trustedServer adapter. Only the SSAT path was traced, so every Prebid render was invisible to the overlay. Instrument the /auction path at its authoritative render signal. The adapter now forwards the server-side trace tuple to Prebid as meta.tsAuctionId / meta.tsAdmHash, and a bidWon listener records an `auction`-path entry (servedFrom: prebid) keyed by the ad-unit code with that auction's own ID, hash and visibility. Only bids carrying meta.tsAuctionId — i.e. the trustedServer seat — are traced, so a client-side bidder's render is never attributed to Trusted Server. The listener only observes and stamps; it does not touch Prebid's rendering. Also fix an overclaim in the SSAT path: with inject_adm_for_testing off there is no adm to inject, which left `injected` unset and let panelStatus fall through to `ok` — claiming a confirmed TS render for a slot TS had merely targeted. Targeting-only renders now report `injected: false`, and `ok` requires an explicit confirmed placement so any future path that omits the signal degrades to gam-only rather than silently claiming success.
On the SSAT proxy path the browser calls /auction against the trusted-server edge domain (e.g. ts.example.com), which was leaking into ext.trusted_server.request_host on the outbound Prebid Server request. That field must track the publisher's own domain instead, matching site.domain/publisher.domain and what PBS's trusted_server verification module expects.
A client framework can replace the ad divs after GPT slots were bound to them: the publisher's React app serves ids like `ad-header-0-_R_ssr_` and swaps them for client ids (`ad-header-0-_r_1_`) during hydration. GPT is left holding slots whose element no longer exists — GAM reports "defineSlot was called without a corresponding DIV", still fetches a creative for them, and the bid is silently wasted with nowhere to render. Waiting for the divs to merely exist cannot help, because at adInit() time the server-rendered divs are present and are only later replaced. So rather than delaying the initial ad request, detect the swap after the fact: a debounced MutationObserver armed after adInit() looks for TS-defined slots whose element left the document and re-runs adInit(), which destroys the orphans and re-binds against the live DOM — reusing the publisher's own slot for that div when they have since defined one. Bounded deliberately, since each re-bind re-requests the affected slots: a 250ms quiet period, a 5s watch window (measured: the swap lands ~2-3s in, so the existing 2s SPA wait would miss it), and at most two re-binds per page load, with the attempt budget shared across the adInit() the watcher itself triggers so it cannot loop. Verified against the live page through the dev proxy: two orphaned slots were detected and re-bound, leaving zero orphans, with all three slots tracing to live client-id elements. (cherry picked from commit e1badbb)
The panel collapsed each slot to a single row with a ×N counter, so a publisher page that refreshes its slots on every render (autoblog's ad-service refreshes from its own slotRenderEnded handler) showed a climbing number instead of what actually happened. Keep an append-only `window.tsjs.renderLog` alongside the per-slot `renders` registry and render it newest-first, one entry per render, with a wall-clock time and a `#N` sequence. The log is trimmed to the most recent entries so a page that refreshes indefinitely cannot grow it without bound; `renders` still collapses per slot for "did this ever render" checks. Also badge every slot that actually shows a creative, not just confirmed TS renders. Gating the badge on `ok` meant production — where inject_adm_for_testing is off and TS only applies GAM targeting — never displayed one at all, since every slot is honestly `gam-only`. The badge now carries its status colour and mark: green ✓ for a confirmed TS render, blue ◐ for gam-only. Slots with nothing on screen (`empty`) or nothing visible (`hidden`) stay unbadged, as there is no creative to label. (cherry picked from commit 013f5e6)
The server-side auction runs once per navigation, but the slotRenderEnded handler read the winning bid out of window.tsjs.bids on every render. That map never changes, so each publisher-driven GAM refresh re-stamped the page-load auction's id, bidder and adm hash and labelled itself `ssat` — claiming a render the server-side auction never produced. A slot refreshed six times over a minute showed six `ssat` rows all pointing at one auction. Scope the claim to the render that actually consumes it: adInit arms a per-slot flag when it applies bid targeting, and the first slotRenderEnded clears it. Later renders are recorded as `gam-refresh` with the stale attribution dropped from both the record and the DOM markers, since GAM re-requested the slot on its own and the returned creative cannot be traced to any Trusted Server auction. These rows are where the client-side /auction path will report real attribution via Prebid's bidWon once that bundle ships; labelling them `ssat` hid that gap instead of showing it. The /auction recording path is unchanged. (cherry picked from commit c4999f7)
Three trace-panel fixes, all surfaced by the gam-refresh rows the previous commit introduced: - Restore the `gam:filled`/`gam:empty` marker on gam-refresh rows. It was gated on `path === 'ssat'`, which hid GAM's own fill signal on exactly the rows where "did GAM fill it this time" is the whole question. - Stop rendering absent attribution as `? · ?` and `auction ?`. An unattributed GAM refresh carries no bidder/hash/auction id by design, so the row now says `no TS attribution` and drops the auction segment rather than looking like a failed lookup. - Give each render a page-global `seq`, shown as `#N` on both the panel row and the on-creative badge so the two point at each other, and mark the row still live for its slot as `◂ current`. The per-slot render count keeps its own `×N`. seq is module-scoped, not stored on window.tsjs, so a re-executed bundle restarts the sequence instead of handing two renders one number. (cherry picked from commit c0811bc)
The visible row text already says "no TS attribution" for a gam-refresh, but the badge title and row hover tooltip still rendered the same absent fields as `?`, which reads like a failed lookup rather than "there is nothing to attribute here by design". Switch both to `—`, matching the `gam_empty ?? '—'` convention the row tooltip already used elsewhere in the same list. (cherry picked from commit b4908cd)
The pbRender bridge has two branches for serving a winning SSAT bid into GAM's Universal Creative: fetch from PBS Cache, or use `bid.adm` directly when present (added by the SSAT inline-creative work, and the only path production actually exercises for bidders that carry markup inline). The PBS Cache branch calls recordBridgeRender after replying; the inline-adm branch replied, fired win/billing beacons, and logged success, but never called it — so this render path, the strongest confirmation signal SSAT has (TS supplies the exact bytes GAM's own creative asked for by name), was invisible to the trace panel. Every SSAT row capped at gam-only even when this branch was serving real creative. Add the missing call, matching the PBS Cache branch's placement. Add regression coverage on both branches — neither had a trace assertion before, so this gap could recur silently on either one. (cherry picked from commit 25316dd)
…re absent Confirmed live on autoblog: Kargo's response carries neither a Prebid Cache UUID nor an `adid`, so hb_adid was omitted for every SSAT bid from that bidder. That broke the pbRender bridge's reverse lookup (adId -> hb_adid) for GAM's Universal Creative postMessage protocol — verified in the browser that GAM sends real 'Prebid Request' messages with real adIds on this account, but window.tsjs.bids[slot].hb_adid was undefined for all three winning bids, so the bridge could never match any of them. Confirmed rendering (injected: true) was structurally unreachable for this bidder regardless of what GAM did. bid.bid_id (the OpenRTB bid's own `id`, always present per spec) already flows through the pipeline unused for this purpose. It is unique per bid instance rather than a creative identifier, but that is exactly what hb_adid needs here: a stable value GAM's Universal Creative echoes back so the bridge can find the winning bid. Add it as the last-resort fallback, after cache_id, the APS renderer's bid id, and ad_id — all three still take priority where present, locked in by test. (cherry picked from commit 281df38)
# Conflicts: # crates/trusted-server-core/src/ec/prebid_eids.rs # crates/trusted-server-core/src/integrations/prebid.rs # crates/trusted-server-core/src/publisher.rs # crates/trusted-server-core/src/settings.rs # crates/trusted-server-js/lib/src/integrations/prebid/index.ts # crates/trusted-server-js/lib/test/build-prebid-external.test.mjs # crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts # docs/guide/integrations/prebid.md
build_auction_request derived publisher.domain, site.domain, and the page URL host from the incoming request Host header. On the SSAT proxy path that header is the trusted-server edge host (e.g. the staging domain), which then leaked into the outbound OpenRTB bid request and, through it, into injected creatives and the IAS brand-safety pixel. Source these fields from settings.publisher.domain instead, matching what convert_tsjs_to_auction_request already does on the /auction endpoint path. Closes #936
…n requests) into rc/july # Conflicts: # crates/trusted-server-core/src/publisher.rs
hb_adid is not unique per bid: absent PBS Cache it falls back to a creative id a bidder may reuse across slots (observed: three IX bids sharing one id). The bridge matched the first bid whose hb_adid equalled the requested adId, then rejected on the slot-ownership guard, so every slot but the first rendered blank. Resolve the bid by the requesting slot and verify its hb_adid matches the request, so each slot renders its own creative regardless of duplicate ids. The adId check still blocks a slot A iframe from pulling slot B's creative and beacons.
The pbRender bridge sized every inline response from the first configured slot format, while the winning creative's own width/height were emitted only inside the testing-only debug_bid. A multi-size slot whose winner is not the first format therefore rendered at the wrong size (clipping or whitespace). Emit w/h in the normal bid map and AuctionBidData, and prefer them in the inline bridge response, falling back to the first slot format only when absent.
The inline render path forwarded adm without resolving the auction-price
macro. URL rewriting then serialized query pairs, encoding the literal
${AUCTION_PRICE} into %24%7BAUCTION_PRICE%7D inside the signed proxy/click
URL — so trackers received an encoded macro rather than the clearing
price, and signing locked the wrong value.
Add expand_auction_price_macro and call it from build_bid_map before
sanitize_creative_html and rewrite_inline_creative_html, using the exact
winning CPM. Only the clear-price token is expanded; the encrypted
${AUCTION_PRICE:B64} variant is left for the DSP.
Brings in the #988 browser-spec fix through its PR lineage and, because #988 stacks on #963's head, refreshes rc's stale #963 absorption with the July 29-30 rework: - bid.meta second descriptor carrier and bidAccepted registration replacing the requestId stash (prebid shim and Rust provider) - Hardened APS auction delivery: sanitized publisher page identity (query/fragment stripped), delivery drop telemetry with dropped_winner_count/reasons, imp disposition counters - ProviderLaunchState/ProviderRequestOutcome orchestrator refactor with parse_state threading and Immediate outcomes - as_aps() Option accessor, fail-closed render-bridge stop, responsive slot-root helpers, case-insensitive APS exclusion tests Preserved rc-only systems the #963 branch predates: #956 opt-in creative processing (process_auction_creative, sanitize_creatives), #967 decoupled prebid shim (public markWinningBidAsUsed instead of prebid.js internals), #948/#912 GPT sync, #865 platform timeout canonicalization (restored at both launch paths and both mediator paths, with the duplicate backend-name pre/post-launch guards and their test suite ported to the new provider API), and the provider-validation startup checks.
PR #916 was squash-merged into main and this PR was retargeted to main, so the previously merged base content re-conflicted without shared history. Every conflicted region resolves to this branch's version, which already contains the base content plus this PR's changes; the only main-side delta adopted inside a conflicted file is the pub(crate) visibility on process_auction_creative. Main's GPT diagnostics overlay (#974) and lint scope (#984) changes merged cleanly.
…c/july Adopts main's finalized #974 GPT diagnostics (standalone ts_console-gated module, finalize_response wiring, updated overlay/store/badges/binding) over rc's earlier absorbed copy, and #984's whole-package eslint gate (eslint . --max-warnings=0). Keeps rc-only content where the two lint passes collided: the #963/#967/#988 test rewrites in the prebid, APS, request, and ad_init suites, the GptSlotHandoff type, and the unexpected-origin-304 guard alongside main's diagnostics finalize call in publisher.rs.
Address review feedback on the shim/bundle seam: - Gate self-init on a loaded Prebid.js API: when the external bundle is missing, skip installRefreshHandler and user ID setup so publisher GPT refreshes keep their targeting instead of being cleared with no auction to refill them; cover the bail-out path with a test - Stamp registered bidder codes (including aliases) derived from prebid.js metadata and validate client_side_bidders against them, retaining module names for audit output - Make installPrebidNpm idempotent per page via a window.__tsjsPrebidShimInstalled sentinel - Validate the window-global bundle manifest shape before use - Warn once about an unstamped User ID manifest instead of once per configured module - Add a processQueue watchdog to the generated bundle entry so pbjs.que still drains if the shim artifact fails to load - Point the missing-adapter error at [integrations.prebid.bundle].adapters and ts prebid bundle - Assert the external bundle script precedes the deferred shim in processed HTML - Add an artifact integration test that builds and evaluates both production outputs together, plus a guard that the shim stays Prebid-free - Delete the unused generated-module placeholders and document the lockstep bundle/server rollout
CodeQL flagged the click guard's navigation and href-persist sinks: the inputs are creative-controlled DOM attributes, so a javascript: value in data-tsclick or href could reach location.href or be written back as an anchor href. Resolve every candidate URL against the pinned trusted base and require an http(s) scheme before navigating or persisting, failing closed otherwise. Also replace an as-any cast in the new click test now that main lints the full JS package (#984).
CodeQL still flagged the href write in persistRebuiltClick: it validated the candidate URL but then wrote the original creative-controlled string. Write the sanitizer's resolved output instead — the http(s)-checked URL absolutized against the pinned trusted base. Beyond closing the taint flow, an absolute href keeps the anchor's default navigation working inside the srcdoc iframe, where a relative value would resolve against about:srcdoc. Tests updated to expect the absolute forms.
- Navigate the observer-repaired click. The mutation observer writes the GET rebuild fallback to href while keeping the canonical signed click in data-tsclick; a later click canonicalized the fallback against that canonical URL, failed the base comparison, and navigated the pre-mutation click. Remember the pending rebuild per anchor and navigate it, and skip no-op attribute writes that would otherwise wake the observer in a loop. - Accept origin-form request targets in the shared signed-target parser. Browsers send /path?query and the Axum adapter forwards it verbatim, so url::Url::parse rejected it as relative — breaking /first-party/click, /first-party/proxy and GET /first-party/sign there, including the second hop of the new rebuild redirect chain. - Inject the click-guard runtime into body-less creative fragments. lol_html matches no <body> in a bare fragment, so common adm shapes shipped without the guard while surviving bidder script could still mutate rewritten links. - Bound rewritten output, not just raw input: rewriting expands every URL into a signed proxy/click URL, so a sub-cap creative could amplify well past it. Reject once the output exceeds the cap. - Fail closed on rewriter errors instead of returning partially rewritten markup, matching the sanitizer. - Treat an explicit empty adm as a supplied creative, not an absent one, so it cannot re-enable the raw PBS Cache fallback without a rejection. - Stamp the first-party origin into the srcdoc document from the parent page and prefer it, then location.origin, over the inherited document.baseURI, which honours a publisher <base> and is not a trustworthy boundary. - Allow proxied assets to load cross-origin. The opaque creative origin makes /first-party/proxy cross-origin, blocking CORS-mode subresources; assets are fetched without client credentials, so a wildcard allow is sound. - Docs: correct the remaining stale auction-orchestration sections, note that #982 only affects deployments with renderGuard enabled, document the cache coordinate policy, and scope the Spin route test's comment to what it asserts.
The srcdoc/opaque-origin conditions the click-guard recovery depends on cannot be reproduced in jsdom: document.baseURI stays an ordinary http URL and window.origin is a real origin, so the about:srcdoc branch and the CORS-blocked POST are never exercised. The Playwright harness already runs in CI, so pin the path there: build the sandboxed srcdoc the client builds (stamped origin, runtime, root-relative signed click), mutate the link, click it, and assert the iframe navigates to /first-party/proxy-rebuild carrying the mutation.
This reverts commit 8589431.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates the July release-candidate changes for review and integration testing before the individual changes merge to
main.Included pull requests
Already included through
mainPending Review / Merge