Skip to content

Dc import info skill - #2130

Draft
rohitkumarbhagat wants to merge 29 commits into
datacommonsorg:masterfrom
rohitkumarbhagat:dc-import-info
Draft

Dc import info skill#2130
rohitkumarbhagat wants to merge 29 commits into
datacommonsorg:masterfrom
rohitkumarbhagat:dc-import-info

Conversation

@rohitkumarbhagat

Copy link
Copy Markdown
Contributor

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a read-only agent support framework (dc-import-info skill) for inspecting Data Commons import extraction and transformation (ET) pipelines. It adds structured operational recipes (GCP Batch, GCS, Logging, Scheduler, Spanner, and local), architectural references, environment configuration defaults, and Python helper scripts with unit tests. Feedback on the changes suggests defensively stripping whitespace from absolute_import_name in the GCS summary-list helper to prevent validation failures, and adding a conditional check in the Scheduler describe-job recipe's jq command to robustly handle missing or null HTTP target bodies.

Comment on lines +59 to +61
def normalize_import_name(absolute_import_name: str) -> dict[str, str]:
"""Validates an absolute import name and derives its exact GCS prefix."""
match = _IMPORT_NAME_PATTERN.fullmatch(absolute_import_name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The absolute_import_name is matched against _IMPORT_NAME_PATTERN without stripping leading or trailing whitespace. If a user or agent passes an input with accidental surrounding whitespace, the validation will fail. Stripping the input defensively ensures robustness.

def normalize_import_name(absolute_import_name: str) -> dict[str, str]:
    """Validates an absolute import name and derives its exact GCS prefix."""
    absolute_import_name = absolute_import_name.strip()
    match = _IMPORT_NAME_PATTERN.fullmatch(absolute_import_name)

Comment on lines +28 to +30
target_uri: .httpTarget.uri,
target_import_name:
(.httpTarget.body | @base64d | fromjson | .argument.importName)}'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The jq expression (.httpTarget.body | @base64d | fromjson | .argument.importName) will throw an error and crash the command if .httpTarget.body is null or missing (e.g., if the Scheduler job is misconfigured or uses a different target type). Adding a conditional check makes the recipe more robust.

Suggested change
target_uri: .httpTarget.uri,
target_import_name:
(.httpTarget.body | @base64d | fromjson | .argument.importName)}'
target_uri: .httpTarget.uri,
target_import_name:
(if .httpTarget.body then (.httpTarget.body | @base64d | fromjson | .argument.importName) else null end)}'

@rohitkumarbhagat rohitkumarbhagat changed the title Dc import info Dc import info skill Aug 4, 2026
rohitkumarbhagat and others added 28 commits August 4, 2026 17:07
…ources with a central configuration file and explicit prompt overrides
…documentation, and remove obsolete import utility scripts
…elate-import-runs implementation with list-import-summaries logic
…esolution with Batch trace, and update contract tests
…dy handling to treat missing bodies as target drift
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant