Build1 distinct publisher3 min readUpdated
A three-node Vault Enterprise cluster crash-looped on OpenShift because restricted-v2 lacks IPC_LOCK. The fix took a custom SCC, a hand-written ClusterRole, a RoleBinding and a pod-spec hint.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
The capability itself is the easy half: add it to a custom SCC and move on. The resolver is what eats the afternoon. OpenShift decides which SCC admits a pod, and when two candidates share a priority it chooses by an ordering the author describes as deterministic but not obvious [13]. In this cluster an unrelated `odf-blackbox-scc` sat at the same default priority of 10, and the init container was admitted under it and failed, even though the `vault` service account was bound to the new `vault-ipc-lock` SCC [14]. The RBAC that every walkthrough tells you to write is necessary and not sufficient [19]. What steers the decision is the container asking for `IPC_LOCK` in its own securityContext, which pushes the resolver toward an SCC that advertises it [15], and the author reports that hint is required even with `defaultAddCapabilities` set, because otherwise the resolver may never reach the SCC you wrote [16].
The two error messages are worth committing to memory, because only one of them mentions Vault. Vault needs `IPC_LOCK` to call `mlockall()` so secrets are not swapped to disk, and exits at startup without it [5]. On the upstream image it gets that far: `mlockall()` returns `EPERM` and the process exits saying it requires the capability [8]. On the Red Hat partner image the binary carries `cap_ipc_lock+ep` from `setcap`, the kernel drops it from the bounding set at exec time, and `exec()` fails with `operation not permitted` before Vault starts [9]. Same cause, unrecognisable symptoms [10]. The author moved to the upstream `docker.io/hashicorp/vault-enterprise:2.0.1-ent` [17].
Count the surface area. Granting a single Linux capability takes four artifacts: the SCC, a ClusterRole that custom SCCs do not get for free [12], a RoleBinding, and the capability entry in Helm values [18]. Two of them are cluster-scoped, which is why the deployment is assembled as three ArgoCD sources, with raw manifests carrying the SCCs, Routes and ConfigMaps the chart cannot cleanly own [4], and why a namespace-scoped team cannot land this alone [22].
That boundary predicts the rest of the writeup. The other three areas the author names are the auto-unseal token flow, Raft quorum during rolling updates, and a reconciliation bug in Vault Secrets Operator [3], and none of them are chart-owned state either: a credential to an external system, consensus that has to survive pod replacement, and someone else's reconcile loop [20]. Delegating unseal to HCP Vault's transit engine is the clearest case, since each pod's path to serving traffic now runs through a secret the cluster does not mint [21]. The claim that this is a straightforward combination of well-documented features [2] holds for each feature read on its own. The traps sit in the seams, and the seams are exactly what no single set of docs owns.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
The author set up a three-node Vault Enterprise HA cluster on OpenShift using HCP Vault as the auto-unseal provider via the transit secrets engine.
The author describes the combination as straightforward on paper and well documented, but in practice a series of traps that took multiple sessions to work through.
The post covers four challenge areas: getting IPC_LOCK right on OpenShift, wiring the auto-unseal token flow securely, managing Raft quorum safely during rolling updates, and working around a reconciliation bug in Vault Secrets Operator.
The deployment is GitOps-managed via ArgoCD using a three-source Helm pattern: the upstream Vault chart, a values file from the repo, and raw manifests for cluster-level resources (SCCs, Routes, ConfigMaps) that Helm cannot cleanly own.
Vault requires the IPC_LOCK Linux capability so it can call mlockall() to prevent secrets being swapped to disk; if the capability is missing, Vault exits at startup.
OpenShift's default restricted-v2 Security Context Constraint does not include IPC_LOCK in its allowed capabilities.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Strong first-hand artifacts, single unverified account
The account is unusually checkable for a blog post: full SecurityContextConstraints, ClusterRole and RoleBinding manifests, the exact Helm values path, verbatim error strings for both images, and a named conflicting SCC with its priority. That is enough for a reader to reproduce the failure and the fix. It is nevertheless one self-reported deployment with no vendor documentation, issue link or second account corroborating the resolver tie-break behaviour, and no OpenShift or chart versions are given, so the evidence cannot be graded higher.
One self-reported deployment
Adoption evidence is limited to a single practitioner's own cluster. There are no user counts, no other organizations described, no benchmark or telemetry, and no indication that the described SCC pattern is in use elsewhere. The underlying components are mature vendor products, but this cluster supplies no data on their uptake.
Slightly understated
The framing is modest relative to what is shown: the title and text promise 'lessons learned' and the body delivers reproducible manifests, exact error strings and an explicit admission that some named problems are only being introduced, not solved. No product superiority, performance or scale claim is made. The mild negative reflects that a broadly applicable OpenShift admission-control gotcha is presented as a personal anecdote, while the one soft spot is asserting resolver tie-break behaviour as general truth from a single cluster.
No disclosed affiliation or sponsorship
The supplied source contains no author affiliation, employer, sponsorship or disclosure statement. The post does recommend HCP Vault Dedicated over self-hosting and quotes its Development tier pricing, but nothing in the material establishes whether the author has a commercial relationship with HashiCorp, Red Hat or the publisher, so any incentive score would be inference rather than observation.
Moderate: reproducible detail, one source
Confidence is capped by cluster structure. The technical mechanics are specific enough that most claims about the author's own environment can be taken at face value, and the derived claims follow directly from the published manifests. But there is one publisher, no vendor or issue-tracker corroboration, no version pinning for OpenShift or the chart, and three of the four announced problem areas remain open in the supplied text.
build
Thirteen tasks green, then "give up (Recommended)" on the one that needed understanding1 distinct publisher
build
One alert, two causes, four green dashboards: the day the stack agreed and was wrong1 distinct publisher
product
IBM says buyers moved June capex to hardware, and its mainframe software stack paid for it1 distinct publisher
security
Akrites switches on in September with 20-odd members and a one-to-10 engineer donation band1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 21, 2026