Build1 publisher3 min readPublished
A workflow GitHub synthesises for Dependabot crashed a cron watchdog three nights running
The watchdog's own death and its one true alert both surface as step 6 failing, so three nights of verifying nothing look identical to the night it caught a daily cron that had been dead for 70.4 days.
The Engineer · Build desk
What happened
- A cron in one of the author's repositories stopped firing on 21 June and went unnoticed for ten weeks, during which blog posts whose publish dates had arrived never appeared on the site.
- The watchdog built to catch that has four scheduled runs and all four are red, with the same run conclusion and the same step-by-step breakdown line for line.
- The 30 August run was red because it filed a correct machine-authored issue; the next three were red because the script crashed in step 5 and verified nothing at all.
- The only change between the two was a repository setting: Dependabot went active, and GitHub began listing an extra workflow that the contents API does not recognise.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint A run history where the alert path and the crash path share a conclusion cannot be used as evidence that the checker ran, so the liveness signal needs a different exit code from the alert.
- exposure Any tool that lists workflows through the API and then reads their files can be killed the same way, and enabling GitHub Pages supplies a second synthetic path to trip it.
- decision Choosing between a whitelist on the executable path and a caught 404 decides whether the next genuinely unreadable workflow file is loud or silently counted as having no schedule.
- precedent GitHub can add active-looking workflow entries with no file behind them and no documentation, so a path field from the workflows API should not be handed to a file read unchecked.
The watchdog walks every workflow the GitHub API returns for a repository and reads each one's YAML off the default branch to pull out its cron expression [14]. It shells out through `subprocess.run(["gh", *args], check=True, ...)`, so any non-zero exit from `gh` raises `CalledProcessError`, and nothing in the collection loop catches it [15]. One of the paths the API handed back had no YAML behind it. `dynamic/dependabot/update-graph` comes back with state `active` and the display name "Dependency Graph", and the contents API answers 404 for it on `master` [11][12]. GitHub synthesises the entry, and the author writes that they could not find it documented as an exception; it appears in both of their public repositories [13].
The repositories are collected in order. The script died on the first one and never reached the second, which was the repository the watchdog existed to watch [16].
All four runs put their evidence in the same two lines: step 5, "Check every scheduled workflow in both public repos", success, then step 6, "File an issue when a schedule has stopped", failure [6]. The author checked the four runs against the API rather than trusting the web view, and they agree to the character [6].
The one honest red produced a machine-authored issue at 19:26:02Z naming `datanika-landing :: .github/workflows/daily-rebuild.yml`, last run on a schedule event at 2026-06-21T09:54:45+00:00, 70.4 days earlier, with a `0 6 * * *` cron that should have fired within 38 hours and a workflow still marked active, so not the 60-day inactivity disable [7][8]. A daily cron dead for 70.4 days is roughly 70 rebuilds that never ran [3]. Twenty-six hours later the same red meant the script had checked nothing [2]. Three of the four runs, 75 percent of the history, show step 6 failing for a reason the step name does not describe [1].
Finding your own exposure is one API call: list the workflows and select the paths that do not start with `.github/workflows/`, and anything printed will 404 on `contents/` [17].
The fix in the post is a whitelist on `.github/workflows/`, because GitHub only ever executes workflows out of that directory. A path outside it cannot be a workflow you own, and whatever prefix GitHub invents next needs no code change [19]. That reasoning holds only while the execution path stays fixed; if GitHub ever runs workflows from somewhere else, the whitelist starts skipping real ones. The author rules out the other repair, catching the 404 and continuing. A 404 on a real `.github/workflows/*.yml` file means a token scope, a rename or an API change, and turning that into "no crons found" would have the watchdog report health from an error [20].
What to watch
- Whether GitHub documents the synthetic workflow entries, or adds a third prefix beyond the Dependabot and Pages ones.
- Whether the watchdog gains a liveness signal separate from its alerting exit code, so a crash and a caught cron stop sharing a conclusion.
- How many other tools that enumerate workflows through the API and then read the files are failing the author's one-line check.