Feature Request / Improvement
Today, long-running writes to PyIceberg risk failing when credentials expire mid-transaction. The Java implementation addressed this by adding a vended-credential refresh mechanism that calls a REST catalog endpoint to obtain new credentials whenever the current ones are within five minutes of expiring.
I propose a similar approach for PyIceberg, delivered in three separate PRs:
- VendedCredential class implementation (largest diff) -> PR ready for review
- Credential refresh support in
PyArrowFileIO -> PR pending review and merge of 1
- Credential refresh support in
FsspecFileIO -> PR pending review and merge of 1
PRs 2 and 3 depend on PR 1, which provides the core vended-credential implementation.
Feature Request / Improvement
Today, long-running writes to PyIceberg risk failing when credentials expire mid-transaction. The Java implementation addressed this by adding a vended-credential refresh mechanism that calls a REST catalog endpoint to obtain new credentials whenever the current ones are within five minutes of expiring.
I propose a similar approach for PyIceberg, delivered in three separate PRs:
PyArrowFileIO-> PR pending review and merge of 1FsspecFileIO-> PR pending review and merge of 1PRs 2 and 3 depend on PR 1, which provides the core vended-credential implementation.