Build1 publisher3 min readPublished Updated
Chrome's own resolver masked the dead router DNS that killed four unattended jobs
On 2026-09-12 four jobs died on getaddrinfo while Chrome browsed normally, because Chrome resolves over DoH and Node and Python ask the router. The answer was a probe that logs one JSON line every five minutes.
The Engineer · Build desk
What happened
- Four unattended jobs failed one after another on the morning of 2026-09-12 with Node fetch and Python getaddrinfo errors, while browsing in Chrome worked perfectly through the same window.
- The macOS unified log showed airportd emitting 4,960 SlowWiFiDnsFailure events in 24 hours, an abnormal rate for a machine whose owner had noticed nothing wrong in the browser.
- The incident ended by hand at 13:09, when the author switched the machine off home Wi-Fi and onto phone tethering.
- After that switch, 37 probe runs recorded zero failures, with lookups under 60ms and connects under 100ms on both IPv4 and IPv6.
- The author's response is a bash script that appends a one-line JSONL snapshot of network state every five minutes, using neither Chrome nor claude, and aiming to finish in under 20 seconds.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- exposure Anything that resolves through the system resolver, including cron jobs, Node services and Python workers, can fail for hours while a human spot-check in the browser reports a healthy network.
- constraint Post-hoc log digging cannot be the diagnostic of record here, because a shorter or lower-frequency failure can age out of the unified log before anyone goes looking.
- decision Monitoring a resolver forces a timeout budget on every call in the probe, since the condition under test is exactly the one that makes a lookup hang.
- capability With a per-interval JSONL line on disk, a failed job can be matched to the line and the nameserver in use at that minute without depending on what the system log kept.
Node and Chrome resolve a hostname by different routes. Chrome runs its own resolver, DoH plus a cache, so the router's rotten DNS never reached it. Node's fetch and Python's socket calls hand the name to the system resolver, which on that morning was the home router [6]. The author of the dev.to post calls it the worst kind of failure: "the network is up, but some processes die anyway" [7].
Node's `fetch failed` appeared 343 times in the same 24-hour window [11]. The four jobs died between 05:01 and 08:41 [10], 42 minutes before airportd's burst first shows up in the log at 05:43 [24]. The author attributes that offset to retention rather than a later onset: configd held entries only from 05:54 onward, while airportd went back to 19:00 the previous evening [9][27].
Averaged across the day the counter fires about 207 times an hour; through the 06:00 to 11:00 window it ran 1,000 to 1,700 and peaked at 1,720 in the ten o'clock hour, roughly 29 a minute [25][3][22]. The counter is named `SlowWiFiDnsFailure`, and slow is how dead DNS got logged [12], so grepping for it means picking a rate threshold before you know what normal looks like. From 05:43 to 12:38 is about six hours and fifty-five minutes, and the sustained five-hour stretch is 06:00 to 11:00 [23].
The script collects gateway, SSID, nameserver, a dig against oauth2.googleapis.com, discord.com and note.com, a dig per configured nameserver, curl over IPv4 and IPv6 against two hosts, and a Python getaddrinfo. It all lands in one line [15][19]. It uses neither Chrome nor claude [14]. The exclusion is the point: the probe has to fail the way the jobs failed, so it calls the system resolver itself [15].
Every timeout is explicit, because a run against dead DNS would hang and miss the next slot [18]. dig gets `+time=2 +tries=1`, curl gets `--max-time 6`, and the Python lookup runs in a thread joined at five seconds [17]. system_profiler needs five seconds by itself just to report the SSID [16]. Run serially, those budgets total about 40 seconds: 6 for the three digs, 24 for the four curls, 5 for the lookup, 5 for the SSID, before any per-nameserver digs [20]. Backgrounding them all together is what keeps a run inside the target [16].
For this to reproduce elsewhere, the browser has to resolve names independently of the system resolver [6], and the tooling is macOS, down to airportd's counter and system_profiler's five seconds [2][16]. The cadence is 288 runs a day, with up to five minutes in which nothing is observed [21][26].
`/etc/resolv.conf` is a symlink, and its mtime still read Aug 15. The author almost read that as three weeks on tethering. The target file, `/var/run/resolv.conf`, had an mtime of 13:09:39, matching the switchover [8].
What to watch
- Whether the probe catches a shorter, lower-rate DNS failure that would fall outside the unified log's retention window.
- Whether the SlowWiFiDnsFailure rate returns after switching back from phone tethering to the home router.
- Whether the probe's 20-second target holds on a run where all three dig targets and both curl hosts time out at once.