feat: Add support for writing bloom filters - #3265
Conversation
|
Thanks for working on this @renaudb. I've noticed that in the |
|
@Fokko the issue is with Bodo, not Boto. Bodo forces |
b86268b to
efd56e8
Compare
|
@Fokko this should be ready for review. |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
|
Pyarrow 25 was released so the test for bloomfilter existence should now pass. |
|
@abnobdoss do you have the ability to reopen the PR? I don't see to have the permission to do it. |
|
Hi @renaudb, I don't believe I can either. @kevinjqliu / @Fokko / @geruh would you be able to re-open this? |
Closes #850
Note: This PR is currently held back by boto requiring
pyarrow<=23.1as bloom filter write support was added in pyarrow 24.Rationale for this change
Add support for writing bloom filters to parquet files. This changes leverages the new
bloom_filter_optionswrite_parquetargument in pyarrow 24.Are these changes tested?
Added tests for the metadata parsing. Added a very basic test for the writing path (there is currently no way to test for the existence of a bloomfilter in a parquet file using pyarrow).
Are there any user-facing changes?
N/A