A dev.to LLM retry pattern randomizes Retry-After waits between half and the full provider-requested time

The post draws a line between a 429 and an overload such as a 529 ^[3]. A 429 is the provider telling the client to slow down ^[2]. An overload is congestion on the provider's side. Slowing down helps the client's standing there, but the recovery timeline is out of its hands ^[3]. The retry ladder then puts both in the retry-freely bucket ^[5]. In the snippet the authors ship, the length of each wait depends only on the attempt number and on whether a Retry-After header came back ^[17].

Retry-After is where the prose and the code disagree. According to the post, when a provider sends that header, "that number wins over your formula" ^[8]. The code converts the header to milliseconds and passes it through the same jitter line as the computed backoff: half the base, plus a random share of the other half ^[10]. A provider that asks for 10 seconds can see the client back after 5 ^[16]. The post's reason for jitter is to stop a thousand failed requests from becoming a thousand synchronized retries ^[22]. That goal survives if the randomness is added above the header value. I'd treat the header as a floor.

Five attempts, the default, means at most four sleeps, with bases of 1, 2, 4 and 8 seconds ^[9][18]. A default run therefore spends between 7.5 and 15 seconds asleep ^[18]. The 30-second cap on a single wait first binds after a sixth failed attempt, so at the defaults it guards a sleep the loop never takes ^[19].

The 60-second deadline is the best idea in the snippet ^[9]. The authors wrote that "unbounded patience is not resilience, it is a hung system with good intentions" ^[12]. Their check runs only in the catch block, before a sleep, and llm.call is awaited with no timeout of its own ^[11]. One slow generation can carry the total past 60 seconds ^[20]. The post itself warns that long generations on loaded infrastructure can exceed any sane client timeout ^[13].

Timeouts get the post's most careful rule. A timeout does not mean the request failed, because the provider may have finished it after the client hung up ^[4]. Before retrying a call whose result triggers side effects, the authors want an idempotency story: a guarantee that the side effect happens once even if the call happens twice. "We learned that one expensively," they wrote ^[6]. They once treated a 429 as fatal, and it cost them a night of production downtime ^[2]. A 200 carrying truncated or malformed content reaches retry logic only if a validation layer feeds it ^[14].

The never-retry list covers 400s, validation errors, authentication failures and content-policy refusals ^[7]. "Retrying a 401 in a loop is how you turn an expired key into a locked account," the post says ^[7]. The snippet hands that list to an isRetryable function. The published excerpt does not include its body, or the circuit breaker's threshold and cooldown values ^[21]. Past its threshold, the breaker opens and fails calls fast without touching the API, then sends a few probe requests after a cooldown ^[15]. The post's case for it is a ten-minute outage, during which ten thousand retrying requests achieve nothing and burn rate limits, latency budgets and queue depth ^[15].