Build1 publisher2 min readPublished
One file read inside a restricted agent pod hands over the LLM key
Admission control and RBAC decide who may fetch a Secret object. Neither is in the path once the kubelet has copied the value into a container, and an agent runtime is where that gap gets expensive.
The Engineer · Build desk
What happened
- A demo gave an agent pod an LLM API key as a projected Secret, then turned on the restricted Pod Security Standard at admission, zero RBAC verbs on Secrets, no service account token, a read-only rootfs and dropped capabilities.
- Inside that container, with no code execution and no privilege escalation, a single file read returned LLM_API_KEY sk-provider-2f9c41e8 next to a GPG_KEY value.
- A third pod in the same namespace, holding no secrets and no permissions, replayed the key to the stand-in provider and got HTTP 200, while a deliberately wrong key came back 401.
- The repo themsquared/k8s-agent-secrets-hardening runs on kind, ships 17 assertions, and takes about 25 seconds to prove the sequence once the cluster is up.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint A control map that lists restricted PSS and no Secret RBAC as the mitigation for credential theft covers the API server path only, so tightening the pod spec further does not narrow what the injected value can be used for.
- exposure The set of things that can spend the key is everything that can read the holder process and reach the provider endpoint, and that set is not the one a Secret permissions review enumerates.
- decision The open question for teams is where the credential lives, and how the pod is hardened does not answer it: the kubelet's write happens before any of the pod's own controls apply.
RBAC decides whether a request to the API server is allowed. A process reading its own environment makes no such request; it only reads what the kubelet already put there. The kubelet fetches the Secret object using its own credentials at pod start and writes the value into the container's environment. The copy is sitting inside the container before the agent process exists [7].
Every control in the hardened stage does verify. A privileged pod in the `agents` namespace is rejected at admission, with PodSecurity naming privileged, allowPrivilegeEscalation, unrestricted capabilities, runAsNonRoot and seccompProfile in a single error [8]. `kubectl auth can-i get secrets` for that service account answers `no`, the Role grants only `get` and `list` on ConfigMaps, and the service account directory does not exist inside the container [9][10]. Five controls, all enforced, and the process's own environment block sits outside every one of them [16].
The provider in the demo is a stand-in that returns 401 for any other credential [6]. The 200 is an authentication result against that stand-in only. The replay went to `http://mock-llm.provider.svc.cluster.local:8088`, an in-cluster service address [14]. Three things have to hold for this to reproduce where you work. The key has to arrive as an environment variable, which in the demo means `envFrom: - secretRef: {name: llm-provider-key}` in the pod spec [13]. Something in the container has to be able to read the environment of the process holding it, which the demo does with a file read against pid 1 [3]. And the pod doing the replay has to reach the endpoint that honours the real key. Restrict that egress and the read still succeeds. The value cannot reach the endpoint that honours it.
Why this bites agent workloads harder than a web app is in the author's own framing. Writing as webofmike, the author said agents "make the gap expensive because an agent is a workload whose whole job is to take instructions from text it did not write" [11]. The same post says its author produced a control map for the July 2026 agent intrusion, one that put this on a checklist item and moved on. The exercise, the post says, is running the item to see what it buys [12].
The demo's third namespace runs agentgateway v1.5.0 [6]. The text published on dev.to breaks off mid-sentence in the explanation of the kubelet's write, before any stage that uses that gateway. It never gets to what brokering the credential buys, and what it does document is cheap to check: 17 assertions on a kind cluster, about 25 seconds once the cluster is up [5].
What to watch
- Whether the gateway stage, once published, shows a credential the agent pod never holds or just another projection of the same key.
- Whether the 17 assertions still hold on a managed control plane instead of kind, where Secret projection is wired differently.
- Whether LLM providers issue short-lived scoped keys that make a replayed value useless from a second pod.