Loading today’s stories
Build1 distinct publisher3 min readUpdated
An EKS cluster refused to consolidate while dashboards showed nodes at 10% CPU. The scheduler was reading resource requests, not utilization, and one workload requested 250x what it used.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
An engineer documenting an Amazon EKS cluster on dev.to found nodes that would not leave the cluster even though their observed CPU and memory looked low, and traced the cause to Kubernetes resource requests rather than to Karpenter itself [1][6]. That matters because the two numbers most operators watch on a dashboard, CPU and memory utilization, are not the numbers the scheduler is allocating against [11].
The sequence is worth following because the first instinct was also wrong. The worker nodes looked over-provisioned relative to what the workloads were using, and the obvious move was to shrink them [2]. The metrics said otherwise: one node sat at roughly 98-100% CPU while memory stayed around 12-15% [3]. The constraint was compute, not memory, so the author went up rather than down, from t3.medium to t3.large [4].
More headroom did not produce better packing. With Karpenter managing worker-node capacity [5], nodes persisted despite low observed usage, and consolidation events were visible during the investigation [6][7]. Rather than reach for the autoscaler's knobs, the author compared actual consumption with configured requests, workload by workload [19].
One workload made the gap legible. Observed usage was about 1m CPU and 17-23Mi of memory, against configured requests of 250m CPU and 250Mi of memory [8][9]. That is roughly 250 times the CPU it was using and about 12 times the memory, reserved on the node whether or not the pod ever touched it [10]. Multiply that across several services with similarly inflated requests [19] and a node reporting 10% CPU and 25% memory on a dashboard [12] can be effectively full from the scheduler's point of view, which is what produces less efficient packing [13].
The discipline in this account is the part worth copying. The available Karpenter changes were all tempting and all available: more aggressive consolidation, different disruption settings, NodePool edits, forcing nodes out [16]. The author's argument is that each of those changes the infrastructure layer before the workload layer is understood, and that if workload requirements do not reflect workload behaviour, tuning the autoscaler hides the problem instead of fixing it [16][18]. The dependency chain runs from workload requests through the scheduler to placement, then to consolidation opportunities, and only then to Karpenter's decisions [17]. The author is explicit that requests are not the only input; Karpenter also evaluates several scheduling and disruption constraints [14]. Requests were simply the one constraint that could be controlled directly [15].
The remediation was not a blanket cut. One class of workload moved from requests of 100m CPU and 512Mi memory, with limits of 800m and 1Gi, to requests of 25m and 64Mi with limits of 300m and 256Mi [21], a 4x reduction in requested CPU and 8x in requested memory [22]. The stated goal was requests that represent the workload while keeping reasonable operational headroom, not the smallest possible numbers, and each workload was reviewed individually [20][23].
What to watch: whether request cuts of that size hold under real peaks, since requests also govern eviction ordering and burst behaviour, and the reduced limits in the example are a separate bet from the reduced requests [21]. Watch, too, for the second-order effect on cost, which is what started this series [24]. If your consolidation stalls, compare requested against used capacity per node before touching the NodePool.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
Engineering Build Notes #2, published on dev.to, documents how Kubernetes resource requests became a scheduling constraint in an Amazon EKS cluster.
The worker nodes appeared to have more capacity than the workloads were using, and the obvious optimization would have been to make them smaller.
One node was consistently running at approximately 98-100% CPU utilization while memory utilization remained around 12-15%; the bottleneck was CPU, not memory.
Instead of downsizing the workers, the author increased the node size from t3.medium to t3.large, removing the CPU constraint.
The cluster was running with Karpenter managing worker-node capacity.
After the CPU bottleneck was addressed, some nodes remained in the cluster even though their observed CPU and memory utilization looked relatively low.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Single self-reported field account with concrete but unverifiable numbers
Every figure comes from one first-person dev.to post about an anonymized production cluster: utilization percentages, one workload's observed usage versus its request block, and a before/after requests example. The numbers are specific and the underlying mechanism (Kubernetes scheduling on requests) is standard, which lifts this above anecdote, but there is no second publisher, no reproducible artifact, no cluster-level before/after measurement, and the supplied text is truncated before any post-change production result. The author's own hedge that other Karpenter constraints were not ruled out limits how firmly the diagnosis can be credited.
One practitioner's production cluster, no wider uptake evidence
Adoption evidence is limited to two disclosures from a single environment: a t3.medium to t3.large node change and a progressive, staging-validated rollout of reduced resource requests. Karpenter and EKS are widely deployed platforms, but this cluster supplies no data on how broadly the described right-sizing practice is used, and no third party, vendor, or survey corroborates it.
Claims sit close to evidence, if anything under-sold
The framing is unusually restrained for the genre: the author flags that requests are only one consolidation constraint, warns against shrinking requests to observed usage, insists on per-workload review rather than blanket values, and presents the earlier node upsizing as a correction to his own first instinct. The '250x' figure that carries the dek is arithmetically backed by the quoted numbers. A small negative gap reflects that the conclusions are hedged more than the concrete evidence would require, offset slightly by the absence of any measured post-change consolidation result to back the implicit payoff.
Mild personal-brand incentive, no vendor or commercial stake disclosed
The piece is a numbered installment in a personal engineering series on a developer blogging platform, which carries a normal reputational incentive to present a clean diagnosis-to-fix narrative and to keep the series going. No product, employer, sponsor, or vendor is promoted, no tool is being sold, and the cluster contains no evidence of commercial arrangement, so the incentive pressure is low-to-moderate rather than material.
Moderate: sound mechanism, thin and single-sourced specifics
Confidence is anchored by the fact that the core technical mechanism - the Kubernetes scheduler reserving against requests rather than live utilization - is well established and internally consistent with the reported figures. It is held down by having one publisher, one anonymized environment, no independent verification, no quantified outcome, and a truncated article body that omits the post-change production results.
build
Rate limit your MCP servers, because a retrying agent turns one error into a billing incident1 distinct publisher
build
Your meter now runs on someone else's machine: signed receipts, fsync, and failing open1 distinct publisher
build
81% of EKS clusters still run the auth method AWS already told teams to leave1 distinct publisher
build
Thirteen tasks green, then "give up (Recommended)" on the one that needed understanding1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 18, 2026