-
Notifications
You must be signed in to change notification settings - Fork 28
feat(tracing): migrate span ingestion to v3 API with string enums #1684
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
saksharthakkar
wants to merge
19
commits into
main
Choose a base branch
from
feat/trace-v3-migration
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+628
−95
Open
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
d13f011
docs: add trace v3 ingestion migration design spec
saksharthakkar 1bd05ad
docs: add trace v3 ingestion implementation plan
saksharthakkar 23e31a3
feat(tracing): add SpanStatus, SpanSource, ExecutionType, VerbosityLe…
saksharthakkar dcef460
fix(tracing): fix ruff E302, consolidate enum imports, add Task 2 TOD…
saksharthakkar a290d07
feat(tracing): update UiPathSpan fields and otel conversion to use St…
saksharthakkar ca0bf0a
feat(tracing): export SpanStatus, SpanSource, ExecutionType, Verbosit…
saksharthakkar 2f61f08
feat(tracing): migrate LlmOpsHttpExporter to v3 ingest endpoint with …
saksharthakkar 0e5526c
fix(tracing): update stale v2 URL in TestUpsertSpan fixture
saksharthakkar 6451b5f
feat(tracing): update LiveTrackingSpanProcessor to use SpanStatus fro…
saksharthakkar aad6b3e
chore(tracing): final lint, type-check and integration test for v3 mi…
saksharthakkar d9222ed
chore(tracing): fix mypy type annotation in integration test, add sho…
saksharthakkar e2c0a55
chore: remove docs from branch (keep locally only)
saksharthakkar f642e7b
Merge origin/main into feat/trace-v3-migration; resolve _span_utils c…
saksharthakkar 3f2f167
fix(tracing): align v3 span payload with backend SpanV3Req; bump vers…
saksharthakkar cf8eba7
fix(deps): bump uipath min pin on uipath-platform to >=0.1.77
saksharthakkar e2e3d73
Merge origin/main into feat/trace-v3-migration
saksharthakkar 8639147
fix(tracing): map GraphInterrupt to Running and mirror full SourceEnum
saksharthakkar 8d7c0c0
fix(tracing): apply bool guard to all int lookups; omit unset Guid keys
saksharthakkar 3bcb72d
fix(tracing): use explicit is-not-None check for status_override
saksharthakkar File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,10 +5,10 @@ | |
| import uuid | ||
| from dataclasses import dataclass, field | ||
| from datetime import datetime | ||
| from enum import IntEnum | ||
| from enum import IntEnum, StrEnum | ||
| from functools import lru_cache | ||
| from os import environ as env | ||
| from typing import Any, Dict, List, Optional | ||
| from typing import Any, Dict, List, Optional, TypeVar | ||
|
|
||
| from opentelemetry.sdk.trace import ReadableSpan | ||
| from opentelemetry.trace import StatusCode | ||
|
|
@@ -17,8 +17,105 @@ | |
|
|
||
| logger = logging.getLogger(__name__) | ||
|
|
||
| # SourceEnum.CodedAgents = 10 (default for Python SDK / coded agents) | ||
| DEFAULT_SOURCE = 10 | ||
|
|
||
| class SpanStatus(StrEnum): | ||
| UNSET = "Unset" | ||
| OK = "Ok" | ||
| ERROR = "Error" | ||
| RUNNING = "Running" | ||
| RESTRICTED = "Restricted" | ||
| CANCELLED = "Cancelled" | ||
|
|
||
|
|
||
| class SpanSource(StrEnum): | ||
| # Mirrors the server's SourceEnum | ||
| # (llm-observability: UiPath.LLMOps.DataAccess/Models/SourceEnum.cs). | ||
| # Member name = exact wire string (no naming policy on the v3 API). | ||
| # Keep complete: an unknown int is relabeled CodedAgents (see | ||
| # otel_span_to_uipath_span), and v3 rejects raw integers. | ||
| TESTING = "Testing" | ||
| AGENTS = "Agents" | ||
| PROCESS_ORCHESTRATION = "ProcessOrchestration" | ||
| API_WORKFLOWS = "ApiWorkflows" | ||
| ROBOTS = "Robots" | ||
| CONVERSATIONAL_AGENTS_SERVICE = "ConversationalAgentsService" | ||
| INTEGRATION_SERVICE_TRIGGER = "IntegrationServiceTrigger" | ||
| PLAYGROUND = "Playground" | ||
| GOVERNANCE = "Governance" | ||
| IXP_UNSTRUCTURED_AND_COMPLEX_DOCUMENTS = "IXPUnstructuredAndComplexDocuments" | ||
| CODED_AGENTS = "CodedAgents" | ||
| IXP_COMMUNICATIONS_MINING = "IXPCommunicationsMining" | ||
| ENTERPRISE_CONTEXT_SERVICE = "EnterpriseContextService" | ||
| MCP = "MCP" | ||
| A2A = "A2A" | ||
| SERVERLESS = "Serverless" | ||
| DOCUMENT_UNDERSTANDING = "DocumentUnderstanding" | ||
|
|
||
|
|
||
| class VerbosityLevel(StrEnum): | ||
| VERBOSE = "Verbose" | ||
| TRACE = "Trace" | ||
| INFORMATION = "Information" | ||
| WARNING = "Warning" | ||
| ERROR = "Error" | ||
| CRITICAL = "Critical" | ||
| OFF = "Off" | ||
|
|
||
|
|
||
| class ExecutionType(StrEnum): | ||
| DEBUG = "Debug" | ||
| RUNTIME = "Runtime" | ||
|
|
||
|
|
||
| # Int→StrEnum lookup tables for converting raw OTEL attribute integers | ||
| _EXECUTION_TYPE_BY_INT: dict[int, ExecutionType] = { | ||
| 0: ExecutionType.DEBUG, | ||
| 1: ExecutionType.RUNTIME, | ||
| } | ||
|
|
||
| _VERBOSITY_LEVEL_BY_INT: dict[int, VerbosityLevel] = { | ||
| 0: VerbosityLevel.VERBOSE, | ||
| 1: VerbosityLevel.TRACE, | ||
| 2: VerbosityLevel.INFORMATION, | ||
| 3: VerbosityLevel.WARNING, | ||
| 4: VerbosityLevel.ERROR, | ||
| 5: VerbosityLevel.CRITICAL, | ||
| 6: VerbosityLevel.OFF, | ||
| } | ||
|
|
||
| _SOURCE_BY_INT: dict[int, SpanSource] = { | ||
| 0: SpanSource.TESTING, | ||
| 1: SpanSource.AGENTS, | ||
| 2: SpanSource.PROCESS_ORCHESTRATION, | ||
| 3: SpanSource.API_WORKFLOWS, | ||
| 4: SpanSource.ROBOTS, | ||
| 5: SpanSource.CONVERSATIONAL_AGENTS_SERVICE, | ||
| 6: SpanSource.INTEGRATION_SERVICE_TRIGGER, | ||
| 7: SpanSource.PLAYGROUND, | ||
| 8: SpanSource.GOVERNANCE, | ||
| 9: SpanSource.IXP_UNSTRUCTURED_AND_COMPLEX_DOCUMENTS, | ||
| 10: SpanSource.CODED_AGENTS, | ||
| 11: SpanSource.IXP_COMMUNICATIONS_MINING, | ||
| 12: SpanSource.ENTERPRISE_CONTEXT_SERVICE, | ||
| 13: SpanSource.MCP, | ||
| 14: SpanSource.A2A, | ||
| 15: SpanSource.SERVERLESS, | ||
| 16: SpanSource.DOCUMENT_UNDERSTANDING, | ||
| } | ||
|
|
||
| _IntEnumT = TypeVar("_IntEnumT") | ||
|
|
||
|
|
||
| def _enum_from_int(table: dict[int, _IntEnumT], raw: Any) -> Optional[_IntEnumT]: | ||
| """Map a raw OTEL int attribute to its enum member, or None. | ||
|
|
||
| Returns None for missing / non-int input — and explicitly for ``bool``, | ||
| since ``bool`` is an ``int`` subclass (``True == 1``) and would otherwise | ||
| be coerced to the value-1 member. | ||
| """ | ||
| if isinstance(raw, bool) or not isinstance(raw, int): | ||
| return None | ||
| return table.get(raw) | ||
|
|
||
|
|
||
| @lru_cache(maxsize=1) | ||
|
|
@@ -65,16 +162,6 @@ class AttachmentDirection(IntEnum): | |
| OUT = 2 | ||
|
|
||
|
|
||
| class VerbosityLevel(IntEnum): | ||
| VERBOSE = 0 | ||
| TRACE = 1 | ||
| INFORMATION = 2 | ||
| WARNING = 3 | ||
| ERROR = 4 | ||
| CRITICAL = 5 | ||
| OFF = 6 | ||
|
|
||
|
|
||
| class SpanAttachment(BaseModel): | ||
| """Represents an attachment in the UiPath tracing system.""" | ||
|
|
||
|
|
@@ -106,20 +193,24 @@ class UiPathSpan: | |
| parent_id: Optional[str] = None # 16-char hex (OTEL span ID format) | ||
| start_time: str = field(default_factory=lambda: datetime.now().isoformat()) | ||
| end_time: str = field(default_factory=lambda: datetime.now().isoformat()) | ||
| status: int = 1 | ||
| status: SpanStatus = SpanStatus.OK | ||
| created_at: str = field(default_factory=lambda: datetime.now().isoformat() + "Z") | ||
| updated_at: str = field(default_factory=lambda: datetime.now().isoformat() + "Z") | ||
| # Default to None (not "") when unset; to_dict() then omits these keys | ||
| # entirely. The v3 ingest endpoint binds them to Guid fields: "" crashes the | ||
| # serializer, and even null fails for the required OrganizationId/FolderKey. | ||
| # In the platform runtime these are always set to real GUIDs. | ||
| organization_id: Optional[str] = field( | ||
| default_factory=lambda: env.get("UIPATH_ORGANIZATION_ID", "") | ||
| default_factory=lambda: env.get("UIPATH_ORGANIZATION_ID") or None | ||
| ) | ||
| tenant_id: Optional[str] = field( | ||
| default_factory=lambda: env.get("UIPATH_TENANT_ID", "") | ||
| default_factory=lambda: env.get("UIPATH_TENANT_ID") or None | ||
| ) | ||
| expiry_time_utc: Optional[str] = None | ||
| folder_key: Optional[str] = field( | ||
| default_factory=lambda: env.get("UIPATH_FOLDER_KEY", "") | ||
| default_factory=lambda: env.get("UIPATH_FOLDER_KEY") or None | ||
| ) | ||
| source: int = DEFAULT_SOURCE | ||
| source: SpanSource = SpanSource.CODED_AGENTS | ||
| span_type: str = "Coded Agents" | ||
| process_key: Optional[str] = field( | ||
| default_factory=lambda: env.get("UIPATH_PROCESS_UUID") | ||
|
|
@@ -131,9 +222,9 @@ class UiPathSpan: | |
| job_key: Optional[str] = field(default_factory=lambda: env.get("UIPATH_JOB_KEY")) | ||
|
|
||
| # Top-level fields for internal tracing schema | ||
| execution_type: Optional[int] = None | ||
| execution_type: Optional[ExecutionType] = None | ||
| agent_version: Optional[str] = None | ||
| verbosity_level: Optional[int] = None | ||
| verbosity_level: Optional[VerbosityLevel] = None | ||
| attachments: Optional[List[SpanAttachment]] = None | ||
|
|
||
| def to_dict(self, serialize_attributes: bool = True) -> Dict[str, Any]: | ||
|
|
@@ -182,9 +273,16 @@ def to_dict(self, serialize_attributes: bool = True) -> Dict[str, Any]: | |
| "JobKey": self.job_key, | ||
| "ReferenceId": self.reference_id, | ||
| "ExecutionType": self.execution_type, | ||
| "AgentVersion": self.agent_version, | ||
| # v3 ingest (SpanV3Req) has no AgentVersion field; the agent version | ||
| # is carried by ReferenceVersion (pairs with ReferenceId above). | ||
| "ReferenceVersion": self.agent_version, | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. cc @jepadil23 are your referencehierarchy changes in yet? |
||
| "Attachments": attachments_out, | ||
| } | ||
| # Omit Guid-typed keys when unset — v3 binds them to Guid columns and | ||
| # rejects null/"". When present (platform runtime) they hold real GUIDs. | ||
| for guid_key in ("OrganizationId", "TenantId", "FolderKey"): | ||
| if result[guid_key] is None: | ||
| del result[guid_key] | ||
| if self.verbosity_level is not None: | ||
| result["VerbosityLevel"] = self.verbosity_level | ||
| return result | ||
|
|
@@ -264,9 +362,9 @@ def otel_span_to_uipath_span( | |
| attributes_dict: dict[str, Any] = dict(otel_attrs) if otel_attrs else {} | ||
|
|
||
| # Map status | ||
| status = 1 # Default to OK | ||
| status = SpanStatus.OK | ||
| if otel_span.status.status_code == StatusCode.ERROR: | ||
| status = 2 # Error | ||
| status = SpanStatus.ERROR | ||
| attributes_dict["error"] = otel_span.status.description | ||
|
|
||
| # Process inputs - avoid redundant parsing if already parsed | ||
|
|
@@ -332,17 +430,33 @@ def otel_span_to_uipath_span( | |
| span_type_value = attributes_dict.get("span_type", "OpenTelemetry") | ||
| span_type = str(span_type_value) | ||
|
|
||
| # Top-level fields for internal tracing schema | ||
| execution_type = attributes_dict.get("executionType") | ||
| # Top-level fields for internal tracing schema. The int->enum lookups go | ||
| # through _enum_from_int so they all ignore bools/non-ints identically. | ||
| execution_type = _enum_from_int( | ||
| _EXECUTION_TYPE_BY_INT, attributes_dict.get("executionType") | ||
| ) | ||
| agent_version = attributes_dict.get("agentVersion") | ||
| reference_id = attributes_dict.get("agentId") or attributes_dict.get( | ||
| "referenceId" | ||
| ) | ||
| verbosity_level = attributes_dict.get("verbosityLevel") | ||
| verbosity_level = _enum_from_int( | ||
| _VERBOSITY_LEVEL_BY_INT, attributes_dict.get("verbosityLevel") | ||
| ) | ||
|
|
||
| # Source: override via uipath.source attribute, else DEFAULT_SOURCE | ||
| uipath_source = attributes_dict.get("uipath.source") | ||
| source = uipath_source if isinstance(uipath_source, int) else DEFAULT_SOURCE | ||
| # Source: override via uipath.source attribute, else CodedAgents. | ||
| # A real int that isn't a known source is relabeled CodedAgents but | ||
| # logged — v3 ingest rejects raw integers, so it can't be forwarded. | ||
| uipath_source_raw = attributes_dict.get("uipath.source") | ||
| source = _enum_from_int(_SOURCE_BY_INT, uipath_source_raw) | ||
| if source is None: | ||
| if isinstance(uipath_source_raw, int) and not isinstance( | ||
| uipath_source_raw, bool | ||
| ): | ||
| logger.warning( | ||
| "Unknown uipath.source int %s; defaulting to CodedAgents", | ||
| uipath_source_raw, | ||
| ) | ||
| source = SpanSource.CODED_AGENTS | ||
|
|
||
| attachments = None | ||
| attachments_data = attributes_dict.get("attachments") | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
todo: we should probably expose these enums as a package so there isn't a copy-paste dependency