Three API calls, no rollback: the phone numbers your carrier bills and your database never saw

A developer who runs a virtual phone number product has published a breakdown of what a purchase flow really does when it calls a carrier, writes a row to its own database, and creates a Stripe subscription in the same request ^[1][5]. The three systems share no transaction log, have no two-phase commit, and none of them can roll back the others, so the tidy snippet is an uncompensated distributed transaction with money on both ends ^[2]. The two directions of leak are simple to state. The carrier keeps invoicing for a number the database has never heard of; Stripe stops invoicing for a number the database still believes is paid up ^[3]. Neither system raises the subject ^[4]. Three sequential writes have two gaps between them, and by the author's account every bad state is reachable from a plain network timeout at a bad moment ^[1][6]. The instinct is to armour the write path with sagas, compensating transactions and idempotency keys everywhere. He recommends the idempotency keys, on the grounds that they cost nothing and do help, while noting the gap does not close: "carrier says yes, then the process dies before the insert" is always reachable, and reordering the calls only changes which orphan you get ^[9]. That is the part worth internalising. The worst class of orphan is invisible from inside your own product: no row, no user, no support ticket, just a number sitting in the carrier's inventory producing an invoice line every month until somebody actually reads the invoice ^[7]. The class where a customer is charged for something the product has released at least generates a complaint; the rest leak quietly ^[8]. His answer is a scheduled reconciler, running daily, designed with three properties ^[10]. Diagnosis is read-only and physically separate from action: one function reads all three systems and returns a delta, other code decides what to do with it, so the diagnosis can be run from a script against production on a Sunday with zero chance of changing anything ^[11]. It acts only when intent is unambiguous. Database says pending_cancellation and Stripe says canceled means a customer.subscription.deleted webhook was dropped, so finalize locally ^[12]. Database says cancelled but the carrier still holds the number means a release call failed, which is rare and where the obvious automated fix is destructive if the diagnosis was wrong, so it logs an anomaly for a human ^[13]. Anything stranger goes into a deltas array for admin review rather than a clever branch ^[14]. And it iterates every provider account: they keep more than one account per carrier, and their first orphan scan walked only the primary, leaving orphan inventory undiscovered in the second account for weeks ^[15]. Two operational details he flags as having paid for themselves. Give the reconciler a real timeout budget, because their run makes four passes with per-row calls out to a carrier, Stripe and an app store, the default serverless function timeout cuts it partway through, the late passes are the money backstops, and a truncated reconciliation looks exactly like a clean one in the logs ^[16]. Alert on deltas rather than fixes, because nobody reads "reconciled 3 rows" every morning for a year, and the signal you want is the thing the reconciler refused to touch ^[17]. The second category of pain is typing. "Phone number" is a family of products that the search API flattens into rows that look interchangeable ^[18]. A result carrying sms: true says the number is technically SMS-capable, not that a message will land, which depends on account configuration, the messaging profile, and in some regions a registration you have not filed ^[19]. They shipped numbers sold as able to receive texts that could not; the flag was not wrong, it answered a different question than the UI was asking ^[20].