Published · 5h agoBuild2 min read
One probe bug, three Kubernetes distributions, and a Go tag nobody opted into
A repro script for kubernetes/kubernetes#141155 shows restarted containers getting liveness probes before their startup probe on k3s, kind and minikube. Distribution choice is not the variable.
Written for builders.See today for builders
What happened
- Kubernetes issue #141155 reports that a container with startup and liveness probes, after both succeed and the liveness probe is flipped to fail, restarts and the new container sees the liveness probe before the startup probe.
- The reporter of #141155 expected the new container to first pass its startup probes before liveness probes begin, citing the Kubernetes startup probe documentation.
- The reproduction script in #141155 accepts --k3s, --minikube or --kind and pins rancher/k3s:v1.36.2-k3s1, kindest/node:v1.36.1 and minikube Kubernetes version v1.36.2, running the same manifest and sequence in each.
- In the repro manifest, the startupProbe has periodSeconds 1 and failureThreshold 100, and the livenessProbe has periodSeconds 1 and failureThreshold 1.
- The configured startup probe allows 100 seconds of failing checks before it gives up, while the liveness probe allows 1 second.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
Three is a count of environments, not of bug reports. The script filed with #141155 takes a flag, boots k3s, minikube or kind from pinned images, and runs the identical sequence in each [3]. Packaging as different as a single privileged k3s container and a kindest/node image landing on the same ordering points at the component they share: the kubelet, which is the thing that actually executes probes [6]. Pinning a different distribution does not route around that.
The manifest in the script shows why ordering carries weight. Its startup probe runs every second with a failure threshold of 100, and its liveness probe runs every second with a threshold of one [4]. That is 100 seconds of protected boot on the first start against one second on the restarted container if liveness gets there first [5]. A startup probe is an admission that the process is slow to come up, and a restart is exactly when it comes up again.
etcd is where this compounds. The etcd liveness probe checks /health three times, the default failure threshold, and restarts the server when all three fail [7]. Issue #96886 argues that restarting fixes almost nothing /health reports: not a NOSPACE alarm, not a missing raft leader, not slow QGETs from an overloaded cluster, which will simply receive the same load again [8]. The same issue cites event-etcd being restarted by the kubelet under overload when leaving it alone would have let it finish and recover [9].
The Go 1.27 case is the same shape at the toolchain layer. The new inline JSON tag is honoured by encoding/json, not only v2, deliberately, so a library adopting it does not force its dependents onto the new API [11]. Roughly 29,000 uses of that tag already exist in public code, much of it Kubernetes-derived from a time when the tag did nothing at all [12][17]. On named rather than embedded fields the count is 2,639, or 694 once openai-go copies are filtered out [13]. The issue author reports no definitively broken case and no confidence that there is none [14].
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Kubernetes issue #141155 reports that a container with startup and liveness probes, after both succeed and the liveness probe is flipped to fail, restarts and the new container sees the liveness probe before the startup probe.
ReportedView cited source - [2]
The reporter of #141155 expected the new container to first pass its startup probes before liveness probes begin, citing the Kubernetes startup probe documentation.
- [3]
The reproduction script in #141155 accepts --k3s, --minikube or --kind and pins rancher/k3s:v1.36.2-k3s1, kindest/node:v1.36.1 and minikube Kubernetes version v1.36.2, running the same manifest and sequence in each.
ReportedView cited source - [4]
In the repro manifest, the startupProbe has periodSeconds 1 and failureThreshold 100, and the livenessProbe has periodSeconds 1 and failureThreshold 1.
ReportedView cited source - [6]
The kubelet manages containers and executes the probes that track application health, and can restart containers to handle some faults.
ReportedView cited source - [7]
The etcd livenessProbe in Kubernetes probes the /health endpoint three times (the default failureThreshold) and restarts the etcd server if all of them fail.
ReportedView cited source
Sources & coverage · 4 publishers
The reporting this story was synthesized from, earliest first. Every link goes to the original.
- kubernetes.io5h agoPod Lifecycle | Kubernetes

