Build1 publisher3 min readPublished
An operator moved seven services onto k3s and back because he wrote none of them
The GitOps tooling worked, and seven services still went back to Docker Compose in a day, because the operator consumes every one of those images and cuts release tags for none of them.
The Engineer · Build desk

What happened
- An operator moved seven self-hosted services off Docker Compose and onto K3s installed on the same 4-core, 24-gig box, alongside the existing compose stacks and sharing the kernel's network stack.
- The cluster ran ArgoCD with an app-of-apps root, a per-service migration template, Kyverno admission policy, Trivy image scanning and SealedSecrets for credentials.
- ArgoCD reported Synced and Healthy for every Application, and the operator then reversed the migration anyway.
- Homepage, trilium, dify, vikunja, apprise and the LLM stack all returned to compose, and so did the k3s-native services that had no compose predecessor.
- The reason given is that every migrated service was an open-source image pulled from an upstream registry, none of it software the operator wrote.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint When the version tag is cut upstream, the delivery controls a mesh and a canary sell you have nothing of yours to act on, and the operator's only lever is running the image or not running it.
- cost Wiring a stock image into network policy, CI, internal DNS and admission rules is labour the estate pays for in hours, and what it buys is legibility for the platform.
- decision The call on offer here is narrower than tearing k3s out: keep the cluster foundation, the GitOps loop and the policy layer for code you own, and stop moving consumed images into them.
The reversal was cheap because of a rule set before the first service moved: external domains and ports stay unchanged, and the reverse proxy stays the anchor [6]. A service comes up on k3s, gets working, and only then does the proxy rule get repointed [6]. Both directions are the same operation. That is why the move up took a few weeks [1] and the move back took a single day [2]. Nothing in the migration failed [9]. That is harder to argue with than an outage.
The case against keeping the services on the cluster is about who owns the version tag. Canary releases, service meshes, mTLS between services and gradual rollouts exist to give an owner fine-grained control over their own delivery, the post argues, and a third-party image arrives with a tag someone else cut, so the only available control is on or off [14]. "The mesh and the canary and the grader were looking for a producer, and I showed up as a consumer," the author wrote [15].
The integration cost is the part I would expect to bite in any small estate. A stock upstream image was not written for your network policy, your CI pipeline, your internal DNS or your admission rules [13]. Either it stays half-integrated, or someone wires the sealed box into the platform, and the post's position is that this second option produces nothing anyone ever uses [13].
The third test is supply-chain policy, and the post grants that a clean scan is a real notion: a scanner runs against every image and flags the CVEs it finds [16]. Trivy was doing that job in the migrated stack [4]. For an image you build, a finding is a change you can make. For one you pull, in my view, it is a wait on upstream plus a policy exception in the meantime.
What came down was one line of the plan, the one that said move existing compose services into k8s [9]. The cluster foundation, the GitOps loop, the supply-chain policy and the service mesh were kept, and the post says most are still running [10]. The workloads left on the cluster are lab-environment, headlamp and a PR-lanes namespace running a two-tier practice app named hello [8]. That is three items, all of them cluster tooling or practice [17].
The stated cause is authorship. "Everything I had migrated was software I didn't write," the author wrote [11], and the list is dify, vikunja, trilium, homepage, apprise and the model runners, all pulled from upstream registries [12]. The 4-core, 24-gig box is the setting [3]. The argument built on it turns on who cuts the version tags, and it does not appeal to team size or coordination cost at all. For the reasoning to transfer, your estate has to look like that one: mostly images whose releases you do not control. Put two services you actually ship on the same box and the mesh, the canary and the rollout have a subject again. The write-up does not report what k3s, ArgoCD, Kyverno and Trivy cost in CPU or memory on that machine [18]. I would want that before repeating either half of the experiment.
What to watch
- The finished supply-chain section, where the post compares scanning images the author writes against images he only pulls.
- Whether any service the author ships himself lands on the box, which would give the mesh, canary and rollout something of his to control.
- Whether the retained k3s foundation keeps running once its only workloads are the lab namespace and headlamp.