Skip to content

Product1 publisher3 min readPublished

After a Helm rename, 46 old Azure DevOps releases could still redeploy the retired path

Forty-six Azure DevOps Classic releases could still redeploy Helm names and image paths retired in a rename, a devops.com account shows. Each kept the task and variable values it was created with, so the cutover is done only once those old copies are found and abandoned.

The Product Desk · Product desk

What happened

  • The author of a devops.com account renamed a set of back-end deployments in nonproduction, moving current definitions, workloads and image repositories to the new names.
  • An older Azure DevOps Classic release still offered Redeploy, and its inline task still held the retired Helm release name and image path.
  • Across five release definitions the author counted 46 stale releases carrying that old task, though nobody had redeployed one.
  • Microsoft describes a Classic release as a snapshot, and editing its definition changes later releases but not the task graph stored in existing ones.

Compiled by The Product DeskSomething wrong?How this is made

Why it matters

  • constraint A correct release definition cannot confirm a Classic cutover is complete; the check has to run against the task stored in every active release.
  • exposure Credentials captured in the 46 stale releases sit out of reach of any variable-group edit, so database access from them stays unproven until each release is checked.
  • exposure A stale redeploy that fails at the ingress would still leave objects under the retired name, including a Service that could stay reachable inside the cluster.

While the old release still held the host and path, the admission webhook refused the new release's duplicate ingress, according to the author's first-person account on devops.com [9]. The refusal came late. Helm had already created a Secret, a ServiceAccount, a Service and a Deployment, and because the task used neither --atomic nor --cleanup-on-fail, all four stayed in the cluster [9]. That happened during the forward rename. The author never ran a stale release, and describes the stale-redeploy path as conditional, based on that failure order [10].

The old releases would follow the same sequence. Their stored task called Helm with --install against a release name that no longer existed, and Helm documents that flag as an instruction to install when it cannot find the name [8]. A redeploy would therefore attempt a second installation under the retired identity [8]. The cutover had changed four coupled values together: the image repository path, the packaged chart name, the Helm release name and the Kubernetes resource names [7]. The current definition showed only the replacements [7].

The habit is to treat the release definition as the pipeline. The author tested how far a release's snapshot reaches: changing a linked variable group did not change an existing Classic release, and only a newly created release picked up the new values [11].

The first containment step was deleting the retired image repositories, once no active workload or current pipeline referenced them [13]. After that, a stale deploy could still create objects, but its pod could not pull the old image and would sit in ImagePullBackOff [13]. The author checked current releases and workloads before deleting anything [14]. Finding the stale releases took the Release API, filtering each definition by active status and maximum creation time [15]. The cutover timestamp only produced candidates. Each release's stored task decided whether it was stale [15].

In my view a Classic migration is finished when no active release holds a stored task that names a retired value. The tradeoff is rollback. Deleting the retired repositories removed image-level rollback through the old path, and an abandoned release is one fewer way back to the old configuration [14]. Sort each old release by two tests: whether its stored task references any retired name or path, and whether the image it points to can still be pulled.

- Retired task, pullable image: a redeploy could start pods whose usefulness depends on the old image and the values captured at creation [12]. Abandon these first. - Retired task, image deleted: a redeploy ends in ImagePullBackOff with orphaned objects [13]. Abandon them, then clean up what they left. - Current task, pullable image: an ordinary release and a working rollback. Keep it. - Current task, image deleted: a repository still in use was removed, the case the author checked for before deleting [14]. Restore it before retiring anything else.

What to watch

  • The stale-redeploy failure path is inferred from the forward rename; an actual redeploy of one of the old releases would confirm or correct it.
  • The outcome of abandoning the 46 stale releases, and whether current workloads come through unchanged.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories