Build1 publisher3 min readPublished
A 30-second Temporal poll replaces six idle GitHub runners with one-job Nomad dispatches
One homelab operator's Temporal workflow polls GitHub every 30 seconds and dispatches one-job Nomad runners, retiring six idle per-repo runners. Each runner stays bound to one repo, but code now mints the tokens and capacity is reserved only while a job runs.
The Engineer · Build desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- The jobs involved push images to a private registry, deploy Nomad jobs and apply Terragrunt for DNS, Vault policies and ACLs, so they must run inside the cluster.
- Outside a GitHub organisation, according to the author, a self-hosted runner can be registered to only one repository.
- The Nomad batch job sets restart and reschedule attempts to zero, so each runner registers, takes exactly one job, deregisters and exits.
- Each runner reserves cpu 2000 and memory 2048 and is constrained to amd64 nodes, which keeps it off the cluster's Raspberry Pi 5s.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Adding a repo still means a separate registration per job because every runner is scoped to one repo; the gain is that a Consul KV entry replaces a hand-registered token.
- cost Jobs pay up to 30 seconds of polling delay plus a forced image pull before the first step, acceptable for deploys and harder to accept for test suites that run on every push.
- capability Because capacity is held only for the life of one allocation, the flat reservation can be sized for the heaviest job and costs nothing between runs.
The one-repo limit survives this design. Every dispatch sets `RUNNER_SCOPE = "repo"` and requires `repo_url` and `runner_token` as meta [8], so each runner still belongs to exactly one repository [3]. The workflow automates the registration step. The HandleRunner child mints a fresh token on every dispatch [6]. Six repos still mean six registrations, but code makes them per job and none outlives its job.
The per-repo count is the careful part. PollAndDispatch lists queued self-hosted jobs, counts runners already pending or running, and starts a child only for each runner that is missing [5]. That count stops a loop firing every 30 seconds from dispatching twice for one queued job [5]. It also handles failure. Restart and reschedule attempts are both zero [7], and the author wrote that "a finished runner should stay finished" [13]. Under the loop as described, a runner that dies before claiming its job drops out of the pending-or-running count while the job stays queued, so the next poll dispatches a replacement [4]. Retries happen in one place, the Temporal loop.
The resource block is where the idle cost goes. The job reserves `cpu = 2000` and `memory = 2048` [9]. In the comment above it, the author notes that because the runner is an ephemeral one-shot, a flat reservation carries no idle cost, so there is no need for memory_max or oversubscription [10]. Had each of the old idle runners carried the same reservation, six of them would have held 12,000 CPU units and 12,288 MB of memory around the clock [1]. I think the flat reservation is the right call for an allocation that exists for one job and then exits.
Start-up latency is the price. A job queued just after a poll waits up to 30 seconds before the scheduler sees it [2]. The runner then pulls its image. `force_pull = true` fetches `:latest` on each dispatch so a rebuilt tool image is never stale, under a ten-minute pull timeout [12]. The post does not report measured queue-to-first-step times. For registry pushes, Nomad deploys and Terragrunt applies [1], I'd accept half a minute of queue time.
The design is cheap on this cluster because most of it was already installed. Temporal ran nightly backups, snapshots and maintenance before it ran CI [4]; once a durable scheduler is on the cluster, most chores start to look like workflows. Per-repo config sits in Consul KV [5]. Vault already exchanges Nomad workload identity for tokens [11]. On bare Nomad, the same result means standing up Temporal, Consul and Vault to retire six idle runners [5].
One line needs checking before anyone copies the job file. A Vault template writes a Nomad ACL token into the task as `NOMAD_TOKEN`, and `NOMAD_ADDR` points at the cluster's Nomad API [11]. The comment scopes that token to a submit-job policy and gives its use as `nomad job validate` and `plan` [11]. Every step of every workflow in those repos runs with the token in its environment. `RUN_AS_ROOT = "false"` [8] restricts the container user and leaves the token's reach unchanged.
What to watch
- Measured queue-to-first-step times under the 30-second poll and forced image pull.
- Whether the author narrows the submit-job Nomad policy to what validate and plan actually need.
- Any change by GitHub to how many repositories a personal-account self-hosted runner can serve.