Skip to content

Build1 publisher3 min readPublished

One kubectl run of nginx:latest tests whether a cluster enforces signed images

k8s-secure-supply-chain, a kind-based reference build, uses five Kyverno checks to refuse at admission any image that CI did not sign and attest. Its author argues that CI scans and signatures stay advisory until the cluster enforces them.

The Engineer · Build desk

Illustration accompanying One kubectl run of nginx:latest tests whether a cluster enforces signed images

What happened

  • A developer published k8s-secure-supply-chain, a kind and Argo CD reference build whose make verify shows a signed image running and unsigned or unapproved-registry images refused.
  • CI builds the image, writes a CycloneDX SBOM with Syft, gates on Trivy, then pushes to ghcr.io and signs the digest keylessly with Cosign using the runner's OIDC token.
  • Kyverno admission checks the approved registry, signer identity, attached SBOM, pinned tag and resource limits before it lets a pod run.
  • A CI job builds an image with 133 fixable HIGH and CRITICAL findings and fails unless the Trivy gate rejects it.
  • The author's proposed test is one command, kubectl run test --image=nginx:latest -n production, and a running pod means nothing is being enforced.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint Once identity and SBOM checks are enforced, upstream images that skipped this pipeline fail admission, so teams running third-party images need a separate way to get them into production.
  • decision Teams have to sign and verify digests: a signed tag proves little when anyone with push rights can move it to a different image.
  • exposure With no stored secrets in the build job, there is no long-lived signing key to leak; what the cluster trusts is the CI runner's per-run OIDC identity.

The author's complaint about most pipelines is that their security steps produce evidence that nothing acts on. "The scan report lands in an artifact folder nobody opens. The signature sits in the registry, verified by no one," the author wrote [14]. The proposed test is a pod built to fail: "A real image, from a registry you never chose, signed by nobody" [1]. "If that pod runs, your supply chain security is a set of suggestions," the author wrote [2].

Measured against the reference policy, that pod fails at least four of the five checks. It comes from an unapproved registry. It has no signature from the expected identity and no signed SBOM. And its latest tag is a mutable pointer [2]. Resource limits depend on how the pod is created.

The design puts trust in one place. The GitHub half and the laptop half meet only at the registry, and the author says that is deliberate [9]. "The cluster never trusts the pipeline. It trusts the evidence the pipeline left behind, and it checks that evidence itself," the author wrote [9]. In ghcr.io, that evidence is the image, its signature, the SBOM attestation and a Rekor log entry [19]. Trivy runs before the push, so the registry only ever holds images that passed the gate [15]. The cluster never has to ask the pipeline whether a scan ran.

Order matters on the cluster side too. Argo CD installs Kyverno in sync wave 0, the policies in wave 1 and the demo app in wave 2 [13]. So the policy engine is up before its policies arrive, and the policies are in place before the first workload they have to judge.

The best craft in the repo is a test of the gate itself. The demo image is built on a minimal Chainguard base with zero known vulnerabilities. That meant the Trivy gate had only ever been seen passing, "which looks exactly like a gate that's switched off," the author wrote [11]. The test job reads the gate's settings straight out of the real build workflow. If someone loosens the gate, the test loosens with it and goes red [12]. That is the job I would copy first. Before any build, CI also runs lint, offline Kyverno policy tests, and gitleaks plus Semgrep across the full git history [17].

All of this runs on a laptop. Terraform creates a local registry and a two-node kind cluster, and `make up` and `make down` build and destroy it [16]. For the `make verify` result to carry over to a real cluster, two things have to hold. The admission checks must cover every namespace where workloads land. And the signer identity the policy expects must be the one CI actually signs with. The author makes a similar point about tooling: "Two engineers can pick identical tools and build very different platforms" [4]. The repo records four design decisions as ADRs. The author also lists three mistakes that surfaced only after the system was working, and says those are what a reviewer should read most [18].

What to watch

  • The three mistakes the author found only after the build worked; they are the most direct record of where this design breaks.
  • Whether the five admission checks hold on a multi-node production cluster that also runs third-party images, beyond the two-node kind demo.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories