Build1 publisher3 min readPublished
Tailscale's default 180-day key expiry ended with the resolver's boot volume on another VM
One Oracle micro VM held Pi-hole, the exit node and the tailnet's only resolver. When its node key came due at 180 days, both clients lost name resolution and every admin path into the box was already closed.
The Engineer · Build desk
What happened
- A dev.to author moved ad blocking onto an Oracle Always Free micro VM in Sao Paulo, running Pi-hole v6 as the resolver for a Windows 11 desktop and a Galaxy S23 joined over Tailscale, with the same box as exit node.
- After six months the desktop and the phone lost name resolution at the same moment, while a ping to 1.1.1.1 still answered normally.
- Tailscale expires node keys after 180 days by default, the VM's key came due, and the node dropped off the mesh without any notice to the clients depending on it.
- Getting back in was blocked three ways: port 22 was closed in the VCN ingress rules, the original SSH private key was misplaced, and the GRUB menu would not come up.
- Recovery meant stopping the micro, attaching its boot volume to a second Always Free A1 instance, mounting the root partition at /mnt/rescue, and appending a fresh public key to authorized_keys.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint A tailnet-wide DNS override with a single entry ties name resolution on every client to one node's key lifetime, and the clients have nowhere else to send a query when that node leaves.
- exposure With 53 and 22 both closed at the cloud firewall, a lapsed node key leaves no remote path to the one machine that has to be fixed.
- decision The boot-volume rescue only exists if a second working instance is already running in the same account, so the out-of-band path is a build-time purchase, not an incident-time one.
- contradiction Oracle's own FAQ covers billing after a Pay-As-You-Go upgrade while the claim that the upgrade stops idle reclamation rests on community threads, so the reclamation risk cannot be read off the documentation.
A ping to 1.1.1.1 needs no resolver, so one test put the fault in the DNS layer [6]. Tailscale's global DNS pushed a single resolver to every device on the tailnet, the VM's own tailnet IP, with override enabled [4]. The author reports that when the VM left the mesh, every query went nowhere [8].
Divide 365 by 180 and you get 2.03. A headless node whose key is renewed only when it expires falls off the mesh about twice a year [23].
The DNS outage was one failure. The lockout was a second, and it came out of the reachability design: port 22 was closed in the VCN ingress rules because the box was only ever reached over Tailscale [9]. Mesh membership was the only credentialed path in, and the expiry closed it.
The exposure hardening here is good work, and it is the same work that removed the recovery paths. Pi-hole was bound to `tailscale0` with `dns.listeningMode=SINGLE`, which the post describes as "permit all origins, accept only on the specified interface" [11]. The first attempt used `ALL`, which accepts on every interface, and the Pi-hole docs call that a way to end up running an open resolver [12]. UDP and TCP port 53 were closed in the OCI security list so that the resolver was reachable only inside the mesh [13].
Same pattern on the client side. To reach the Windows 11 desktop with plain RDP over the tailnet, the author disabled "Only allow Windows Hello sign-in for Microsoft accounts" and used a local admin account, which re-enables password authentication and weakens the machine; the stated mitigation is a long unique password with NLA left on [19]. Tailscale SSH is the tighter option when the full desktop is not needed, according to the post [20].
The expiry itself is documented. It sits on Tailscale's Linux install page, expiry section included, which the author read when the box was set up [14]. "I've seen people describe this as hidden. It isn't. I skipped it," the author wrote [15].
A second clock runs on the same VM. Oracle reclaims idle Always Free instances, and defines idle as CPU 95th percentile under 20%, network under 20% and memory under 20% for A1 shapes, measured over a 7-day window [16]. A resolver plus exit node clears those thresholds comfortably, the author writes [17]. Oracle's FAQ says Always Free resources are not charged after an upgrade to Pay-As-You-Go, and community threads say the upgrade also stops reclamation; the docs do not state the second part, and the author calls it "folklore with a good track record, not a guarantee" [18].
In my view the cheap correction is a second entry in the tailnet DNS config, since the post records exactly one [8]. The disk surgery is the expensive one, and it worked only because a second Always Free A1 instance was already running in the account [21]. The alternative the author names is rebuilding the micro from scratch [24].
What to watch
- Whether the rebuilt micro is registered with key expiry disabled or with a renewal on a calendar; the 180-day default otherwise repeats the outage.
- Whether a working SSH key for the micro is escrowed outside the mesh, given that the original private key was misplaced.
- Whether Oracle's documentation ever states that a Pay-As-You-Go upgrade stops idle reclamation, which currently rests on community threads.