Dc import info skill - #2130
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a read-only agent support framework (dc-import-info skill) for inspecting Data Commons import extraction and transformation (ET) pipelines. It adds structured operational recipes (GCP Batch, GCS, Logging, Scheduler, Spanner, and local), architectural references, environment configuration defaults, and Python helper scripts with unit tests. Feedback on the changes suggests defensively stripping whitespace from absolute_import_name in the GCS summary-list helper to prevent validation failures, and adding a conditional check in the Scheduler describe-job recipe's jq command to robustly handle missing or null HTTP target bodies.
| def normalize_import_name(absolute_import_name: str) -> dict[str, str]: | ||
| """Validates an absolute import name and derives its exact GCS prefix.""" | ||
| match = _IMPORT_NAME_PATTERN.fullmatch(absolute_import_name) |
There was a problem hiding this comment.
The absolute_import_name is matched against _IMPORT_NAME_PATTERN without stripping leading or trailing whitespace. If a user or agent passes an input with accidental surrounding whitespace, the validation will fail. Stripping the input defensively ensures robustness.
def normalize_import_name(absolute_import_name: str) -> dict[str, str]:
"""Validates an absolute import name and derives its exact GCS prefix."""
absolute_import_name = absolute_import_name.strip()
match = _IMPORT_NAME_PATTERN.fullmatch(absolute_import_name)| target_uri: .httpTarget.uri, | ||
| target_import_name: | ||
| (.httpTarget.body | @base64d | fromjson | .argument.importName)}' |
There was a problem hiding this comment.
The jq expression (.httpTarget.body | @base64d | fromjson | .argument.importName) will throw an error and crash the command if .httpTarget.body is null or missing (e.g., if the Scheduler job is misconfigured or uses a different target type). Adding a conditional check makes the recipe more robust.
| target_uri: .httpTarget.uri, | |
| target_import_name: | |
| (.httpTarget.body | @base64d | fromjson | .argument.importName)}' | |
| target_uri: .httpTarget.uri, | |
| target_import_name: | |
| (if .httpTarget.body then (.httpTarget.body | @base64d | fromjson | .argument.importName) else null end)}' |
…ources with a central configuration file and explicit prompt overrides
…e GCS artifact naming conventions
… schema extraction logic
…documentation, and remove obsolete import utility scripts
…or import automation validation
…tegration tests for flag validation
…elate-import-runs implementation with list-import-summaries logic
…and documentation
…ounding and status definitions
…atus query recipes to spanner directory
…esolution with Batch trace, and update contract tests
…y-list sentinel documentation
…arify recipe documentation conventions
…gest recovery for mutable image tags
…diness checks and documentation
dd1da30 to
ded1c47
Compare
…dy handling to treat missing bodies as target drift
No description provided.