Skip to content

SDAP-543: Migrate to uv for environment and dependency management#119

Open
MarcCruzs wants to merge 7 commits into
apache:developfrom
MarcCruzs:SDAP-543
Open

SDAP-543: Migrate to uv for environment and dependency management#119
MarcCruzs wants to merge 7 commits into
apache:developfrom
MarcCruzs:SDAP-543

Conversation

@MarcCruzs

Copy link
Copy Markdown

Caveat: Built and tested locally only. Two surfaces remain unverified:

  1. Behavior in an AWS deployment
  2. The config_operator Docker image build (only collection_manager and granule_ingester images were built and tested locally)

Summary

Migrates the ingester repo from its mixed legacy packaging (setuptools + requirements.txt for collection_manager/config_operator/common, and Poetry for granule_ingester) to a single uv workspace with one shared lockfile. All component Docker images are rebuilt on uv.

Supersedes the packaging approach from SDAP-511 (Poetry) and builds on SDAP-542 (Python 3.11).

What changed

Packaging & dependencies

  • Added a root pyproject.toml defining a [tool.uv.workspace] with four members: common, collection_manager, config_operator, granule_ingester.
  • Added a per-component pyproject.toml for each member
  • removed the old setup.py / requirements.txt files and granule_ingester/poetry.lock.
  • Single shared uv.lock at the repo root is now the source of truth for all resolved dependencies (replaces the per-component requirement files and the Poetry lock). Chose this route to reduce the maintenance needed to maintain dependencies.
  • Pinned Python via .python-version and requires-python = ">=3.11,<3.12" across all components.

Docker

  • All three Dockerfiles (collection_manager, config_operator, granule_ingester) rebuilt on uv using a two-phase uv sync (deps first for layer caching, then workspace packages). All are built from the repo root so uv can resolve the workspace.
  • Entry points updated to run out of the uv-managed /app/.venv.

Package names (aligned with the previous published names)

  • common is published as sdap_ingester_common and
    collection_manager as sdap_collection_manager, matching the
    PACKAGE_NAME values from the prior setup.py files. config_operator and
    granule_ingester keep their names. Import module names are unchanged
    (import common, import collection_manager).

config_operator

  • main.py updated for kopf 1.x on Python 3.11 (HandlerFatalError
    PermanentError)
  • kopf pinned to >=1.36,<2.

Documentation & housekeeping

  • READMEs for all components updated to the uv workflow (uv sync --package …,
    uv run --package …, repo-root docker build).
  • Added .gitattributes and a root .dockerignore.
  • CHANGELOG entry under [Unreleased].

Verification

  • uv lock --check passes (75 packages, no drift).
  • collection_manager image: builds from repo root
    • sdap-ingester-common installs
    • import common works
    • main.py -h runs.
  • granule_ingester image: builds (Python-from-source + nexusproto)
    • sdap-ingester-common, nexusproto install, import common, import nexusproto, and main.py -h all succeed.
  • config_operator image: not yet built/tested locally.

Notes

  • config_operator's Repository URL is carried over verbatim from the prior repo metadata.
  • CI workflows under .github/workflows/ are intentionally left unchanged in this PR.
  • For Poetry migration to uv, utilized uvx migrate-to-uv (https://github.com/osprey-oss/migrate-to-uv)

MarcCruzs added 7 commits June 8, 2026 18:49
  - Add .dockerignore; add .gitattributes (eol=lf) — CRLF broke the
    granule install_python shebang on Linux.
  - config_operator: kopf 0.26->1.42.5 + HandlerFatalError->PermanentError,
    bump kubernetes, install git for GitPython (3.11 compat).
  - granule_ingester: point UV_PYTHON at the venv so nexusproto installs
    there; override protobuf 3.2.0->3.20.3 for 3.11.
  - Re-lock uv.lock.
uv.lock, collection_manager, common, config_operator, and
granule_ingester components. Then updated README to reflect the uv
implementations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant