[bot] Merge master/357532ea into rel/dev#1688
Conversation
Metric-skill and conversation evaluators create metrics in a shared,
persistent eval workspace but never removed them. A leftover metric is
detected and reused by a later test (the agent returns
`SELECT {existing_id}` instead of full MAQL), so the assertion fails —
and it makes some conversation metric-turns produce no create_metric
call at all.
Delete each metric a run creates on the way out, keyed on the exact
metric_id from the create_metric tool result, in a finally block:
- metric_skill: per-run cleanup in _execute_single_metric_run.
- conversation: collect metric_ids across turns, delete after the
conversation completes (deferred so a later turn's $ref still resolves).
Mirrors the existing alert_skill._delete_alert pattern. Parallelism-safe:
each run deletes only the id it created, so concurrent tests sharing the
workspace never touch each other's objects. No version bump.
JIRA: QA-28379
risk: nonprod
fix(gooddata-eval): delete metrics created during agentic eval runs
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## rel/dev #1688 +/- ##
===========================================
+ Coverage 77.91% 78.02% +0.11%
===========================================
Files 271 271
Lines 18603 18637 +34
===========================================
+ Hits 14494 14542 +48
+ Misses 4109 4095 -14 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
2 participants
🚀 Automated PR to perform merge from master into rel/dev with changes up to 357532e (created by https://github.com/gooddata/gooddata-python-sdk/actions/runs/29232016785).