Skip to content

[net-685] last applied assignment added#66

Open
toschoosqd wants to merge 3 commits into
mainfrom
net-685-last-assignment
Open

[net-685] last applied assignment added#66
toschoosqd wants to merge 3 commits into
mainfrom
net-685-last-assignment

Conversation

@toschoosqd

Copy link
Copy Markdown
Contributor

Summary

Implements the ping-collector portion of [NET-685] behind the mvcc-chunks feature flag.

  • Stores last_applied_assignment_id from worker heartbeats.
  • Adds Nullable(Int64) to fresh worker_pings_v2 tables.
  • Migrates existing tables idempotently with explicit column placement.
  • Preserves missing values as NULL.
  • Rejects non-positive assignment IDs.
  • Documents append-only, max-per-worker consumption semantics.

Dependencies

  • NET-686: defines the strictly monotonic assignment ID contract.
  • sqd-network PR: adds optional last_applied_assignment_id to Heartbeat.
  • The sqd-network commit reference must be updated after that protocol PR is committed.
  • Worker-side NET-685 changes must populate the heartbeat field.
  • NET-681 / NET-684 consume the ClickHouse signal downstream.

Feature Flag

Enabled with:

mvcc-chunks

Legacy builds retain the existing ClickHouse row shape and behavior.

Validation

  • Legacy collector tests: 4 passed, 1 ClickHouse test ignored.
  • Feature-enabled tests: 7 passed, 1 ClickHouse test ignored.
  • Legacy and feature-enabled pings-collector checks passed.
  • sqd-messages tests passed.
  • git diff --check passed.

@toschoosqd toschoosqd requested a review from define-null June 22, 2026 15:27
@toschoosqd toschoosqd self-assigned this Jun 22, 2026
Comment thread crates/collector-utils/src/storage.rs Outdated
Comment thread crates/collector-utils/src/storage.rs

@define-null define-null left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM, left a bunch of suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants