Loading today’s stories
Build1 distinct publisher2 min readUpdated
Every step in the badge pipeline succeeded, so learners ran the lab twice more
An Isovalent Labs engineer instrumented complaint rate and repeat-completion rate, then fixed a working async flow with one slide of copy. No API, webhook or service was touched.
The Engineer · Build desk
What happened
- Users of the Isovalent Labs platform regularly complained that they had not received their badge after completing a hands-on lab, although the badge had been issued; they expected it in the wrong place.
- The flow: Instruqt sends a webhook when a lab is completed, the platform catches and processes the event and asks Credly to issue the credential, and Credly then emails the learner a link to claim it.
- Every part of that chain was working, but the webhook is asynchronous and can arrive after the lab has ended, at a point the UI can no longer catch it, so the expected link was not on the final screen.
- Some users complained, and others completed the lab again, sometimes more than twice, apparently hoping another completion would make the badge appear.
- The author began measuring two things: how often people complained about a missing badge, and how often they repeated a lab after completing it.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
Pipeline telemetry could not have found this defect. Instruqt fired on completion, the platform processed the event, Credly issued the credential, Credly emailed the claim link [2], and every one of those steps reported success [3]. A fault that lives in the gap between when an asynchronous callback lands and when the final screen is still in front of the learner leaves no failed span to find, because nothing failed [3].
The repeats are the part worth staring at. A learner who sits the same exam a second and a third time [4] is paying real minutes to retry a process that already worked. Each of those completions fires the completion webhook again [2], so the issuance side booked additional successful issuances for people who still held nothing they could use [13]. The healthiest-looking counter was being inflated by the confusion it was structurally unable to see.
Which is why the two things the author chose to measure matter more than the fix. Complaint rate about missing badges and repeat rate after completion [5] both sit on the far side of the system boundary, where the actual failure was. They are cheap, they need no new service, and they are the only signals in this story that ever moved in the wrong direction.
The intervention was a slide before the final exam saying that successful learners would get an email from Credly with their badge [6]. Nothing inside the chain was modified: not the webhook, not the API, not the deployment set [7], leaving lab content as the only edited artifact [14]. Complaints fell sharply and most of the unnecessary repeats went away [7].
Read that as a mechanism, not an effect size. The account, published on dev.to by the engineer who ran it [16], offers "fell sharply" and "most" and no baseline or post-change figures for either metric [15]. One platform, one change, no numbers.
The accounting error underneath is more portable. The credential system treated issuance as the end state and therefore recorded success [8], but the learner's sequence runs complete, issue, notify, claim, then use or share, and stopping at issuance is closer to calling a package delivered when it has only left the warehouse [9]. Nobody on the platform side can force a claim or a share [10]. What they can change is what the learner knew beforehand, which is the whole argument for treating divergent user behaviour as a property of the process rather than a defect in the user [12]. The follow-up was direct claim links in later emails, added because reducing uncertainty had moved the binding friction to finding the link [11].
Loading topic controls
Loading related stories