feat(git): skip snapshot upload when HEAD is unchanged#395
Conversation
78ed43f to
8a10581
Compare
8a10581 to
a17628a
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a17628a87c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
When the last completed generation captured the same mirror HEAD recently enough (snapshot-max-age, default 24h) and the cache entry still exists, skip tar/zstd/upload so the cached bytes and ETag stay stable. A stable ETag keeps in-flight snapshot streams and If-Match chunk reads valid across generation intervals when nothing changed. The commit is recorded on the shared coordination record at completion. Skips record a checked-at timestamp instead of a completion: max-age keeps tracking the last actual upload, while peers stay suppressed for the interval so an unchanged repo gets one fetch-and-check per interval rather than one per replica. Mirror snapshots use HEAD as a proxy for the whole mirror: non-HEAD ref drift can be omitted for up to snapshot-max-age, which restored pods cover by fetching immediately. Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019f85b8-d346-7243-961f-fbf2236a7f94
a17628a to
d50fce0
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d50fce039c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return false | ||
| } | ||
| rec, ok := c.gens.Get(snapshotGenKey(job, upstreamURL)) | ||
| return ok && rec.Commit == commit && !rec.CompletedAt.IsZero() && c.now().Sub(rec.CompletedAt) < maxAge |
There was a problem hiding this comment.
When a snapshot job runs just before snapshot-max-age elapses, this still returns true; the generator then skips the upload and records CheckedAt, causing Claim to suppress every replica for another full snapshot interval. During that suppression window the cached snapshot can be served older than the configured max age (and can even expire if max age is near the backend TTL), so the max-age bound no longer actually bounds unchanged snapshots. Consider only skipping when the last upload will remain within maxAge until the next check, or cap CheckedAt freshness at CompletedAt + maxAge.
Useful? React with 👍 / 👎.
2 participants
When the last completed generation captured the same mirror HEAD recently enough (
snapshot-max-age, default 24h) and the cache entry still exists, snapshot generation skips the tar/zstd/upload entirely. The cached bytes and ETag stay stable, so in-flight snapshot streams andIf-Matchchunk reads remain valid across generation intervals when nothing changed — overnight ETag collisions drop to ~zero.The captured commit is recorded on the shared coordination record at completion. A skip releases its claim instead of recording a completion, so the max-age bound keeps tracking the last actual upload and peers aren't suppressed for the full claim TTL. A lost cache entry (crash, expiry, manual delete) fails the existence check and regenerates immediately.
Applies to base, LFS, and mirror snapshots. Mirror snapshots use HEAD as a proxy for the whole mirror: non-HEAD ref drift can be omitted for up to
snapshot-max-age, which restored pods cover by fetching immediately on startup.