Build1 publisher3 min readPublished
The gateway returns 504 while the agent keeps spending tokens
A dev.to walkthrough argues long agent runs need their own run resource, durable state and idempotency keys. Its own sample handler shows how easily the dedupe key gets minted server-side and stops working.
The Engineer · Build desk
What happened
- In the scenario the article opens with, a frontend POSTs to /agent-runs and waits, the agent outlasts the gateway timeout, and the client gets a 504 while the backend worker keeps executing.
- The user retries after the error, which can leave two agents doing overlapping work with no clean way to establish which one is authoritative.
- The prescribed shape is to stop running agents inside the request: create a run record, return 202 Accepted, and execute the work asynchronously.
- The same piece argues agent state has to be stored durably, in a section built around in-memory state dying on deploy, restart or scale-in.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost The asynchronous shape adds a run table, a queue, a worker, a progress channel and a cancellation protocol, and teams whose runs comfortably finish inside the gateway timeout pay for machinery they never exercise.
- decision Who generates the idempotency key stops being a backend detail and becomes a client contract, because a key the server mints cannot survive the retry it exists to collapse.
- exposure Side effects set the severity of a timeout: a read-only run wastes tokens, while one holding write access to mail, orders or records leaves state that nobody has reconciled.
- constraint Once run state is the source of truth, the progress stream is no longer sufficient on its own, so every client needs a queryable run resource and the agent loop needs a cancellation check it actually honours.
The interesting line in the source's sample handler is the fallback. It reads the `Idempotency-Key` header, and when the header is absent it substitutes `crypto.randomUUID()`, then passes that value into `startAgentRunOnce` as the dedupe key [9]. A key the server mints per request dedupes exactly one request: the one that minted it. The retry after a 504 arrives as a fresh request, takes a fresh UUID, and starts a second run [3]. The checklist above the code says idempotency keys prevent duplicate runs [6]. The default in the code says only callers who send a stable key get that, and nothing in the handler tells them they have to.
That makes key provenance a frontend contract. The client generates the key before the first attempt, persists it across a reload, and sends the same value on retry. Otherwise the run table faithfully records both agents.
Why the request abstraction fails is visible in the two lifecycles the piece lays side by side. HTTP gets five steps, connect through close [11]. The agent run gets eight states before it terminates plus three terminal outcomes, so eleven in total [13]. Three of the eight are waiting on something outside the process: the model, a tool, a human approving [14]. Blocked time rather than compute time is what runs past the gateway, and blocked time is when your process is most likely to be replaced by a deploy or removed by a scale-in, which is where in-process run state goes [8].
What the piece does not supply is a threshold. It says agent runs often take seconds, minutes, hours or even days [4], and its 504 lands two minutes in as an illustration rather than a measurement [18]. So the number that decides this locally is yours: the p99 of your run duration against the timeout your gateway is actually configured with, and whether any tool in the plan writes. A read-only run that times out is annoying; a run that can send messages, create orders or delete records is an incident waiting to happen, in the source's framing [15]. If nothing writes, you can keep the synchronous handler and an honest cap. If anything writes, the run record buys the one thing a timed-out handler cannot give you, which is a row you can query to find out what already happened [16].
One adoption cost arrives after the queue is working. Once the run outlives the request, credentials captured at request time can expire while the agent is still thinking [10], so the worker needs a refresh path or a delegated token. That is a change to the auth model, not to the queue.
What to watch
- Whether agent frameworks start shipping a client-side idempotency key convention by default instead of leaving key provenance to the caller.
- Published gateway and platform timeout defaults for agent hosting, which the source never names.
- Whether durable execution engines absorb the human-approval pause so application code stops owning resume logic.