Build1 distinct publisher3 min readUpdated
A CloudNativePG admission webhook timeout on GKE Autopilot had nothing to do with Postgres: the control plane could not reach the webhook, and no operator setting fixed it.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
A team deploying a PostgreSQL cluster with the CloudNativePG operator on GKE Autopilot hit an admission webhook timeout that stopped the cluster from ever being created [1]. The interesting part is where the fault was not: the problem was first seen with EnterpriseDB Postgres for Kubernetes, but the authors report the cause was not specific to EDB or to PostgreSQL at all, and instead came down to how Kubernetes control-plane traffic interacts with restrictive NetworkPolicies on Autopilot [2][3].
The error message points squarely at Postgres, which is why this burns time. The API server returns a failure calling webhook `mcluster.cnpg.io`, with a `dial tcp <webhook-ip>:9443: i/o timeout` against the CNPG webhook service [5]. Everything an operator would normally check was fine: the webhook Pod was running, the Service was healthy, DNS resolved, and other Pods in the cluster connected to the webhook without trouble [6]. One client could not reach it, and it was the Kubernetes API server [7]. That is the diagnostic signature worth memorising: a webhook that is healthy and reachable from application Pods while the API server reports timeouts [13].
The environment was GKE Autopilot on Kubernetes v1.35 with the CNPG operator and admission webhooks enabled [4]. The same deployment worked on a GKE Standard cluster [8], which is the shape of an infrastructure difference rather than a workload bug. Google's own documentation tells you to allow control-plane traffic using the cluster's `masterIpv4CidrBlock` when writing firewall rules for admission webhooks [9]. On Autopilot the control plane is fully managed by Google and that control-plane IP range is not exposed to the customer in the same way [10], so the documented recipe does not have an ingredient you can obtain.
The dead ends are instructive because they are the ones most people try first. Moving the webhook port from 9443 to 443 did not help; raising the webhook timeout from 10 seconds to 30 did not help; reviewing and updating firewall rules did not help [11]. Tripling the timeout budget changes nothing when there is no path at all [17]. Setting `failurePolicy: Ignore` did clear the symptom, and the authors are explicit that this is not a production answer [12], which is correct: it converts a blocked admission call into an unvalidated one, on the resource type least suited to skipping validation.
The blast radius is wider than one database. The writeup lists the CNPG operator, EDB Postgres for Kubernetes, applications built on CloudNativePG, other Kubernetes workloads using admission webhooks, and any workload protected by restrictive NetworkPolicies on Autopilot [14]. Any operator you install into a namespace with a tight policy is exposed to the same class of failure, and it will surface as that operator's bug.
What to watch: the authors say they took this to the Google Cloud Support networking team [15], and the article states it covers the NetworkPolicy change that resolved the issue [16]. The text supplied to this desk breaks off mid-sentence at exactly that point, before the policy itself is shown [18], so treat the specific rule as unverified until you read it. In the meantime the actionable part is the triage order: when a custom resource creation times out on a webhook, test reachability from the control plane separately from reachability from Pods, and compare Autopilot against Standard before touching the operator.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
Google Cloud documentation recommends allowing control-plane traffic using the cluster's masterIpv4CidrBlock when configuring firewall rules for admission webhooks.
With GKE Autopilot the Kubernetes control plane is fully managed by Google and the control-plane IP range is not exposed to the customer in the same way.
After exhausting common troubleshooting approaches, the authors worked with the Google Cloud Support networking team to understand why the Kubernetes API server could not reach the admission webhook.
The article states that it covers, among other things, the root cause and the NetworkPolicy change that resolved the issue.
While deploying a PostgreSQL cluster using the CloudNativePG (CNPG) operator on GKE Autopilot, the authors encountered an admission webhook timeout that prevented the PostgreSQL cluster from being created.
The issue was initially observed with EnterpriseDB (EDB) Postgres for Kubernetes, but the investigation showed the problem was not specific to EDB or PostgreSQL.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed single-cluster field report, no artifacts or corroboration
The account is specific and internally coherent: named platform and Kubernetes version, verbatim API server error including the 9443 dial timeout, an explicit list of components verified healthy, an elimination sequence of failed remediations, and a stated resolution confirmed by the workload deploying. That is well above anecdote. But it is one publisher, one cluster, and one team: no NetworkPolicy manifest or diff, no packet-capture output despite packet-level testing being claimed, no link to the Google Cloud documentation cited, no upstream CloudNativePG/GKE issue or support case reference, and the root-cause language is hedged ('it may not be possible to explicitly reference them'). Nothing independent confirms the mechanism.
Two deployment data points from one team
Adoption signal is limited to the authors' own environment: one blocked CNPG/EDB Postgres deployment on Autopilot and one unblocked deployment after the policy change, plus a passing reference to the workload succeeding on GKE Standard. No second operator, no usage numbers, no maintainer or vendor acknowledgement, and no evidence that the claimed wider blast radius across webhook-using workloads has actually been hit elsewhere.
'Root cause and production fix' framing runs ahead of what is shown
Modestly overstated. The title promises root cause and a production fix, and the cluster framing asserts the API server could not reach the webhook, yet the mechanism is presented with hedged language, no manifest or capture is published, and the accepted remedy is the removal of restrictive traffic selectors - a loosening of a security control offered without any compensating-control or risk discussion, which is a weaker thing than a production fix. The generalisation to 'other Kubernetes workloads using admission webhooks' is extrapolation from a single cluster. The reporting is otherwise sober: failed attempts are listed, and failurePolicy: Ignore is explicitly disowned as non-production, which keeps the gap small rather than large.
Visibility-driven republication, no commercial ask
The piece is a practitioner write-up republished from the IBM Community to dev.to explicitly 'to make the content accessible to a wider technical audience' - a reach and professional-credibility incentive rather than a sales one. It sells no product, and it is mildly unflattering to two named vendors' platforms (GKE Autopilot's abstracted control plane, and the initial EDB/CNPG symptom), while crediting Google Cloud Support. That mix keeps incentive pressure low but non-zero: the narrative arc rewards a clean 'we found the root cause' ending, which plausibly explains the confident title over hedged technical detail.
Plausible mechanism, single unverified account
Moderate confidence in the practical takeaway - that restrictive NetworkPolicy selectors can block Google-managed control-plane traffic to a webhook on Autopilot, and that the GKE Standard comparison plus the API-server-only symptom is a good triage path. Confidence is held down by total dependence on one uncorroborated source, the absence of the fix artifact, hedged root-cause wording, an unverified generalisation to other webhook workloads, and a ledger inconsistency about the article's own contents.
build
Rate limit your MCP servers, because a retrying agent turns one error into a billing incident1 distinct publisher
build
Your "Index Only Scan" Did 2,847 Heap Fetches: Covering Indexes Are a Vacuum Problem1 distinct publisher
build
Your meter now runs on someone else's machine: signed receipts, fsync, and failing open1 distinct publisher
build
The optional EntityManager is the bug: moving the transaction boundary into AsyncLocalStorage1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 18, 2026