Skip to content

Build1 publisher3 min readPublished

Kyverno pins stateless pods to the zone the StatefulSet's volume already chose

AWS EBS volumes are zonal, so a StatefulSet pod can only run in the zone that holds its disk. An admission policy reads that zone and places eligible stateless pods beside it, for a claimed saving above 10%.

The Engineer · Build desk

Illustration accompanying Kyverno pins stateless pods to the zone the StatefulSet's volume already chose

What happened

  • A post on dev.to argues that deliberately co-locating an application's pods with its StatefulSets inside one Availability Zone can cut infrastructure cost by more than 10%, depending on internal traffic and architecture.
  • When the single Postgres, queue or cache instance sits in the zone that fails, an app tier spread across three zones still fails for every user, because each pod depends on that one backend.
  • The implementation is a Kyverno ClusterPolicy that looks up the labelled anchor pod's live zone at admission and injects the placement constraint on eligible stateless pods only.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • cost The ceiling on this saving is whatever share of the bill the zone-crossing line items already occupy, so a cluster where cross-AZ transfer is 3% of spend cannot recover 10%.
  • decision Every stateful workload now needs an explicit answer to whether it is genuinely replicated across zones, because where the replication is real the co-location case stops applying.
  • capability Doing the zone lookup at admission keeps co-location workable across many tenants; hand-maintained nodeAffinity per Deployment goes stale each time an anchor pod is recreated.

Zonal block storage is the constraint doing the work in this argument. A StatefulSet pod backed by an EBS or PD volume can only ever run in the Availability Zone where that volume lives [6]. Kubernetes cannot move it, and a scheduling rule cannot migrate it; affinity only places other pods relative to it [7].

So the availability question is about replication, not spread. Absent a real cross-AZ setup (a Postgres streaming replica, multi-AZ Kafka, a multi-zone Elasticsearch cluster), the workload already sits inside one zone [8]. Lose the zone holding the only database, queue or cache instance and an app tier spread over AZ-1, AZ-2 and AZ-3 fails for all users, because every app pod depends on that one backend [9]. A three-zone Deployment in front of one unreplicated cache has three zones of transfer charges and one zone of uptime. Pinning those pods into the anchor's zone does not meaningfully change real availability in that case, and it removes what the pods paid to reach the backend while the backend was healthy [10].

According to the post, the price of true HA is roughly 99.5% single-AZ against 99.99% multi-AZ, about 1.8 days of potential downtime a year against about an hour, and it applies only to the components actually replicated across zones [12][13]. The arithmetic checks out: 0.5% of 8,760 hours is 43.8 hours, and 0.01% is 53 minutes [20].

The cost claim is where I would want numbers. The post says more than 10% off infrastructure cost, conditioned on internal traffic volume and architecture [3]. The line items it names are pods talking to a StatefulSet in another zone, volumes sitting in a different zone from the pods using them, chatty internal APIs at volume, monitoring and logging traffic spread for no reason, and replication crossing zones with no availability requirement behind it [4]. None of that leaves the region, and most providers bill it as regional data transfer anyway [5]. For removing those items to take more than a tenth off total spend, they have to be more than a tenth of spend before you start; where cross-AZ transfer is 3% of the bill, 3% is the ceiling [21]. No baseline bill and no measured before-and-after appear in the post [22].

The manual version of this is nodeAffinity or nodeSelector, which the post calls static and per-workload [14]. Applied by hand across many tenants it becomes four standing chores: find each anchor's current zone, hardcode it into every related Deployment, update it whenever the anchor pod is recreated elsewhere, and keep it off PVC-backed pods and DaemonSets [15]. The ClusterPolicy named pin-stateless-pods-to-anchor-az does the lookup at pod admission instead, injecting the constraint only for eligible stateless pods [16][18]. Namespaces opt in through a generic tenant label, resolved at admission by an apiCall against /api/v1/namespaces/{{request.namespace}} [19]. The anchor itself is chosen by labelling whichever StatefulSet pod should hold the zone, instead of matching a name pattern [17].

One line decides when any of this takes effect: `background: false`, commented in the policy as mutating only newly-created pods, never retroactively [18]. Nothing moves when you apply it. The saving shows up as pods are recreated, so an opted-in namespace keeps paying until its Deployments have been rolled [23].

What to watch

  • A measured before-and-after on a real bill, with the regional data transfer line broken out, would settle whether the >10% figure transfers.
  • What the policy does when the labelled anchor pod is recreated in a different zone and stateless pods are already pinned to the old one.
  • Any change to intra-region cross-AZ transfer pricing at AWS or another provider, which is the charge this whole pattern removes.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories