Support zstd blob decompression in Puffin - #3575
Merged
Merged
Conversation
ebyhr
marked this pull request as draft
June 28, 2026 11:10
Fokko
pushed a commit
that referenced
this pull request
Jun 30, 2026
# Rationale for this change Fix `_payload` storing `puffin[8:]` instead of `puffin` — blob offsets in the footer are file-relative (from byte 0), not relative to byte 8. The two bugs cancelled for single-blob files at offset 4; the fix was only testable once PuffinFile supported multi-blob/compressed files. Also widen `PuffinBlobMetadata.type` from `Literal["deletion-vector-v1"]` to `str` so `PuffinFile` can parse files containing non-DV blobs without a Pydantic validation error. ## Are these changes tested? Yes. Copied files from https://github.com/apache/iceberg/tree/main/core/src/test/resources/org/apache/iceberg/puffin/v1 `sample-metric-data-compressed-zstd.bin` will be added separately in #3575 ## Are there any user-facing changes? No <!-- In the case of user-facing changes, please add the changelog label. -->
ebyhr
force-pushed
the
ebi/puffin-compression
branch
from
June 30, 2026 04:18
8c7db31 to
4ab4d3f
Compare
ebyhr
force-pushed
the
ebi/puffin-compression
branch
from
June 30, 2026 04:37
4ab4d3f to
e882ba8
Compare
ebyhr
marked this pull request as ready for review
June 30, 2026 05:10
Member
Author
|
Thanks for your review! Addressed the comment. |
Contributor
|
Sorry for letting this linger for a while @ebyhr thanks for working on this! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Support zstd blob decompression in Puffin to enhance its specification coverage.
Are these changes tested?
Yes. Copied a Puffin file from https://github.com/apache/iceberg/tree/main/core/src/test/resources/org/apache/iceberg/puffin/v1
Are there any user-facing changes?
No.