Add workflow that fails when Dependabot updates error#208
Conversation
Dependabot version update failures only surface as a red triangle in the Dependabot tab, which nobody checks. This weekly scheduled workflow fails if any "Dependabot Updates" run concluded with failure, startup_failure, or timed_out in the last 8 days, so a broken ecosystem surfaces as a red scheduled run that emails a human. See maxmind/device-android#61 for the reference implementation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughA new GitHub Actions workflow runs weekly or manually, checks recent “Dependabot Updates” runs for failure-like conclusions, reports failures with metadata, and exits unsuccessfully when any are found. ChangesDependabot failure monitoring
Estimated code review effort: 2 (Simple) | ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Checkov (3.3.8).github/workflows/dependabot-failure-watcher.ymlTraceback (most recent call last): Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds a scheduled GitHub Actions workflow that monitors recent “Dependabot Updates” workflow runs and fails if any have concluded with failure, startup_failure, or timed_out, ensuring Dependabot breakages surface via a failing scheduled run (and associated notifications) rather than only via the Dependabot UI.
Changes:
- Introduces a weekly scheduled workflow (Wednesdays) to query recent Dependabot Updates runs via
gh run list. - Fails the watcher workflow when any Dependabot Updates run in the last 8 days ended unsuccessfully (including
timed_out).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| since=$(date -u -d '8 days ago' +%Y-%m-%dT%H:%M:%SZ) | ||
| failures=$(gh run list \ | ||
| --repo "$REPO" \ | ||
| --workflow "Dependabot Updates" \ | ||
| --limit 100 \ | ||
| --json conclusion,createdAt,displayTitle,url \ | ||
| --jq "[.[] | select((.conclusion == \"failure\" or .conclusion == \"startup_failure\" or .conclusion == \"timed_out\") and .createdAt >= \"$since\")]") | ||
| count=$(echo "$failures" | jq 'length') |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/dependabot-failure-watcher.yml:
- Line 33: Update the gh run list command for the “Dependabot Updates” workflow
to include the --all option, ensuring runs from disabled workflows are
considered while preserving the existing failure-watching behavior.
- Around line 3-11: Update the explanatory comments in the workflow to qualify
the email notification claim: state that a failed scheduled run may email the
relevant user only when their Actions notifications are enabled, and that the
recipient can change when the cron is edited or the workflow is re-enabled.
- Around line 30-40: Update the Dependabot failure query in the workflow to
request and filter on updatedAt instead of createdAt, so runs are selected by
completion time; also report updatedAt in the jq output timestamp while
preserving the existing failure-status and 8-day filtering behavior.
- Line 34: Remove the --limit 100 cap from the workflow’s run lookup and update
the Dependabot failure search to consume all runs at or after since, using
pagination or an equivalent exhaustive retrieval approach. Preserve the existing
8-day lookback and qualifying-failure filtering.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 0ae06917-2050-4a73-bbf9-86da85845189
📒 Files selected for processing (1)
.github/workflows/dependabot-failure-watcher.yml
| # Dependabot version updates run as GitHub Actions workflow runs named | ||
| # "Dependabot Updates". This scheduled job looks back over the past week for any | ||
| # of those runs that failed and fails itself if it finds one, so a silently-broken | ||
| # ecosystem surfaces as a red scheduled run instead of only a red triangle in the | ||
| # Dependabot tab that nobody checks. | ||
| # | ||
| # Runs entirely within this repo (no external service). A failed scheduled run | ||
| # emails the person who last edited the cron below. Note: GitHub auto-disables | ||
| # scheduled workflows after 60 days of repo inactivity. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== workflow file ==\n'
cat -n .github/workflows/dependabot-failure-watcher.yml
printf '\n== nearby references ==\n'
rg -n "email notifications|workflow notifications|scheduled workflow|Dependabot Updates|cron" .github/workflows .github -S || trueRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 3993
🌐 Web query:
GitHub Actions notifications for workflow runs email enabled scheduled workflow recipient cron edited re-enabled docs
💡 Result:
GitHub Actions workflow notifications, including those for scheduled runs, are managed via your personal notification settings [1][2]. To enable email notifications for GitHub Actions: 1. Navigate to your GitHub notification settings (Settings > Notifications) [2][3][4]. 2. Under the System section, locate the Actions setting [2][4]. 3. Change the dropdown from "Don't notify" to "Email" (or "On GitHub" for web notifications) [2]. 4. Optionally, select "Only notify for failed workflows" to reduce noise [2][5][4]. 5. Click Save [2][4]. For scheduled workflows (cron jobs), notifications are sent to a specific recipient based on the following priority rules [1][6]: - Initial creation: Notifications are sent to the user who initially created the workflow [1][7]. - Modified cron syntax: If a user updates the cron syntax in the workflow file, that user becomes the recipient for subsequent notifications [1][8][7]. - Re-enabled workflows: If a scheduled workflow is disabled and then re-enabled, notifications are sent to the user who re-enabled the workflow, overriding the previous recipient [1][6][7]. These settings ensure that only the relevant user receives updates for automated tasks without requiring broad repository-wide notification configurations [1][2].
Citations:
- 1: https://docs.github.com/en/actions/concepts/workflows-and-actions/notifications-for-workflow-runs
- 2: https://docs.github.com/en/subscriptions-and-notifications/how-tos/managing-github-actions-notifications
- 3: https://docs.github.com/subscriptions-and-notifications/get-started/configuring-notifications
- 4: https://www.guideflow.com/tutorial/how-to-manage-github-actions-notifications-in-github
- 5: https://docs.github.com/en/enterprise-server@3.17/subscriptions-and-notifications/how-tos/managing-github-actions-notifications
- 6: https://docs.github.com/en/enterprise-cloud@latest/actions/concepts/workflows-and-actions/notifications-for-workflow-runs
- 7: https://github.com/github/docs/blob/main/content/actions/concepts/workflows-and-actions/notifications-for-workflow-runs.md
- 8: https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
import pathlib, re
p = pathlib.Path('.github/workflows/dependabot-failure-watcher.yml')
text = p.read_text()
print("has scheduled workflow comment:", 'emails the person who last edited the cron below' in text)
for i, line in enumerate(text.splitlines(), 1):
if 1 <= i <= 80:
print(f"{i:3}: {line}")
PYRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 2182
Qualify the notification claim. A failed scheduled run can email the relevant user, but only if their Actions notifications are enabled, and the recipient can change when the cron is edited or the workflow is re-enabled.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml around lines 3 - 11, Update
the explanatory comments in the workflow to qualify the email notification
claim: state that a failed scheduled run may email the relevant user only when
their Actions notifications are enabled, and that the recipient can change when
the cron is edited or the workflow is re-enabled.
| since=$(date -u -d '8 days ago' +%Y-%m-%dT%H:%M:%SZ) | ||
| failures=$(gh run list \ | ||
| --repo "$REPO" \ | ||
| --workflow "Dependabot Updates" \ | ||
| --limit 100 \ | ||
| --json conclusion,createdAt,displayTitle,url \ | ||
| --jq "[.[] | select((.conclusion == \"failure\" or .conclusion == \"startup_failure\" or .conclusion == \"timed_out\") and .createdAt >= \"$since\")]") | ||
| count=$(echo "$failures" | jq 'length') | ||
| if [ "$count" -gt 0 ]; then | ||
| echo "::error::$count failed Dependabot update run(s) in the last 8 days:" | ||
| echo "$failures" | jq -r '.[] | "- \(.displayTitle) (\(.createdAt))\n \(.url)"' |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== workflow file ==\n'
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,160p'
printf '\n== gh run list help excerpts ==\n'
gh run list --help | sed -n '1,220p' || true
printf '\n== search for createdAt/updatedAt usage ==\n'
rg -n '"(createdAt|updatedAt)"|createdAt|updatedAt' .github/workflows . || trueRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 4888
🌐 Web query:
GitHub CLI gh run list JSON updatedAt completion time createdAt semantics
💡 Result:
In the GitHub CLI (gh), the gh run list command provides structured output via the --json flag, which supports the fields createdAt and updatedAt [1][2]. These fields correspond to the timestamps of the workflow run as managed by the GitHub Actions API. Semantics of Timestamps: The createdAt field represents the date and time when the workflow run was initially created or triggered [3]. The updatedAt field represents the date and time when the workflow run was most recently modified [3]. Implementation Details: Within the GitHub CLI codebase, these fields are mapped to time.Time types [3]. When you specify --json createdAt,updatedAt in your command, the CLI retrieves these values from the underlying GitHub API response and serializes them into the JSON output [4]. As these are standard Go time.Time fields, they are typically serialized in ISO 8601 format (e.g., "2026-07-22T10:00:00Z"). The available fields for use with --json in gh run list are explicitly defined as: attempt, conclusion, createdAt, databaseId, displayTitle, event, headBranch, headSha, name, number, startedAt, status, updatedAt, url, workflowDatabaseId, and workflowName [1][5][2].
Citations:
- 1: https://cli.github.com/manual/gh_run_list
- 2:
gh run listshould support--pr <num>cli/cli#10386 - 3: https://github.com/cli/cli/blob/trunk/pkg/cmd/run/shared/shared.go
- 4: Add
--jsonexport flag for workflow runs cli/cli#3869 - 5: Improve
run listdoc with available--jsonfields cli/cli#8934
Filter by completion time, not creation time. createdAt misses runs that started before the 8-day boundary and failed within it. Use updatedAt for the filter and when reporting the run timestamp; gh run list --json exposes that field.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml around lines 30 - 40,
Update the Dependabot failure query in the workflow to request and filter on
updatedAt instead of createdAt, so runs are selected by completion time; also
report updatedAt in the jq output timestamp while preserving the existing
failure-status and 8-day filtering behavior.
| since=$(date -u -d '8 days ago' +%Y-%m-%dT%H:%M:%SZ) | ||
| failures=$(gh run list \ | ||
| --repo "$REPO" \ | ||
| --workflow "Dependabot Updates" \ |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- file ---\n'
sed -n '1,220p' .github/workflows/dependabot-failure-watcher.yml
printf '\n--- locate gh usage ---\n'
grep -RIn --line-number --fixed-strings 'gh run list' .github/workflows . || trueRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 259
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- file ---'
sed -n '1,220p' .github/workflows/dependabot-failure-watcher.yml
printf '\n%s\n' '--- locate gh usage ---'
grep -RIn --line-number --fixed-strings 'gh run list' .github/workflows . || trueRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 2144
🌐 Web query:
gh run list --workflow by name disabled workflows --all docs
💡 Result:
When using the gh run list command, the --workflow (or -w) flag allows you to filter runs by a specific workflow. By default, when you provide a workflow name to this flag, the command will not include runs from disabled workflows [1][2]. To include disabled workflows when filtering by name, you must also pass the --all (or -a) flag [1][3][2]. Usage example: gh run list --workflow "Workflow Name" --all Providing the --all flag ensures that the command fetches runs even if the associated workflow is currently in a disabled state [1][3]. Alternatively, if you identify a workflow by its filename (e.g., "workflow.yml"), the command may fetch runs regardless of the workflow's active status, though using the --all flag is the explicit and recommended approach when filtering by name [4].
Citations:
- 1: https://cli.github.com/manual/gh_run_list
- 2:
gh run listshould support--pr <num>cli/cli#10386 - 3: feat: add
-aflag togh run listcli/cli#9162 - 4: Undocumented behaviour of the
-wflag forgh run listcli/cli#9038
Include --all here. gh run list --workflow "Dependabot Updates" skips runs from disabled workflows unless --all is passed, so a recent failure can be missed and the watcher may report success. Add --all or query the workflow’s run history directly.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml at line 33, Update the gh
run list command for the “Dependabot Updates” workflow to include the --all
option, ensuring runs from disabled workflows are considered while preserving
the existing failure-watching behavior.
| failures=$(gh run list \ | ||
| --repo "$REPO" \ | ||
| --workflow "Dependabot Updates" \ | ||
| --limit 100 \ |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the workflow and nearby context.
git ls-files .github/workflows/dependabot-failure-watcher.yml
wc -l .github/workflows/dependabot-failure-watcher.yml
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,220p'Repository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 2332
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find how the run list is queried and whether pagination or filters are used.
rg -n --no-heading --context 3 "gh run list|--limit|since|failure|dependabot" .github/workflows . || trueRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 3371
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Search for the workflow's trigger cadence and the script body around the limit.
sed -n '1,220p' .github/workflows/dependabot-failure-watcher.ymlRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 1930
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Count how many runs are even plausible within the lookback window by inspecting schedule/branches if present.
python3 - <<'PY'
from pathlib import Path
import re, sys
p = Path('.github/workflows/dependabot-failure-watcher.yml')
text = p.read_text()
print(text)
PYRepository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 1931
🏁 Script executed:
#!/bin/bash
set -euo pipefail
cat -n .github/workflows/dependabot-failure-watcher.yml | sed -n '1,220p'Repository: maxmind/MaxMind-DB-Reader-ruby
Length of output: 2231
Do not cap the lookback at 100 runs. --limit 100 can silently miss a qualifying Dependabot failure within the 8-day window once there are more than 100 runs; paginate or otherwise consume all runs at or after since.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/dependabot-failure-watcher.yml at line 34, Remove the
--limit 100 cap from the workflow’s run lookup and update the Dependabot failure
search to consume all runs at or after since, using pagination or an equivalent
exhaustive retrieval approach. Preserve the existing 8-day lookback and
qualifying-failure filtering.
Add a weekly scheduled workflow that fails when any "Dependabot Updates" run concluded with
failure,startup_failure, ortimed_outin the last 8 days, so a silently-broken Dependabot ecosystem surfaces as a red scheduled run that emails a human instead of only a red triangle in the Dependabot tab that nobody checks.This is a copy of the reference change in maxmind/device-android#61, adjusted to run on Wednesdays and to also treat
timed_outas a failure.🤖 Generated with Claude Code
Summary by CodeRabbit