Build1 publisher3 min readPublished
Offset pagination makes every page request pay for a count over the tenant's whole table
A dev.to account traces missing adgroup data for a platform's largest clients to the count query behind a page index, and its case that AI agents need stable cursors rests on the same four-year-old incident.
The Engineer · Build desk
What happened
- A Friday afternoon message from customer support reported data missing for some clients but not others, an escalation the team had seen recur without knowing what triggered it.
- Reviewing endpoint latency and the adgroup data led to pagination, where the count query needed to compute the page index, not the size of the result set, carried the cost.
- The affected clients had not grown suddenly; they had accumulated enough data for an old architectural assumption to become expensive.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Per-request cost tracks each tenant's accumulated rows, so the endpoint stays fast in staging and for small accounts while failing first for the accounts holding millions of line items.
- decision Accepting the ORM's page-based pager is a bet that no tenant's table keeps growing, and the bet gets placed during the zero-to-one phase by whoever wired up the list endpoint.
- exposure In ad-tech the paged sync sits on the bidding path, so a slow page loop becomes stale serving data and the make-goods that follow it.
- contradiction The post argues cursors matter more because of AI, but the failure it can actually show is row growth, which was already breaking the API four years before agents were the consumers.
Producing a page index requires a total, and the account puts the cost there. Page-based iteration scales linearly while cursor-based paging is constant time, the author writes [6]. Re-running the ORM's long-running queries sometimes returned result sets of 15 million records or more, and the author says the size of the result set was not the real problem [5]. The count behind the page index is proportional to those rows, and it runs on every page request [24].
The same code path therefore costs a different amount per tenant. The objects here are adgroups in The Buyer Cloud, called Beeswax when the original post was written, each holding an advertiser or brand, line items and creative references [18]. What works for clients with a few hundred line items does not work for clients with millions [20].
Most ORMs page by offset because the strategy "requires no assumptions about the underlying table structure", and most frameworks are built to take an app from zero to one quickly [7]. That is also what makes it expensive later, because an application "can unevenly grow out of this strategy over time" [8].
The AI half of the argument arrives as a preference: the post says agents consuming APIs "want stable, deterministic cursors" [15], and that "in the age of A.I., this rings more true now than ever" [16]. The incident underneath it is older. The author first wrote the finding as a journal entry some four-plus years ago [17]. The account demonstrates a row-growth failure, and the agent case rests on that same behaviour at least four years after it first bit [23].
For the linear count to hurt your API, two things have to hold. Your list endpoint has to issue a count on every page request, and at least one tenant's row count has to keep growing without a cap. If the count is cached, or tenant data is bounded, the linear term never gets big enough to trigger an alert. In this case both held, and the sync had a deadline: adgroup data must reach serving infrastructure for live bidding, where the author ranks "seconds are great, minutes are acceptable, hours can result in disaster and costly Make Goods" [19].
A team can adopt the detection half without a migration. The post's prescription is growth-based observability, watching per-client data growth before latency spikes [21]. The tooling in the account is ordinary: Datadog for endpoint latency, plus the AWS RDS panel for long-running queries, where Django's generated SQL ran from 200 to more than 1,000 lines [11][12]. Before the diagnosis, the issue surfaced in various forms once or twice a month, with per-client frequency and priorities anywhere from P1 to P4 [13], which is 12 to 24 escalations a year [22]. The author's gloss on calling it random is "a word used when engineering hasn't quite figured out the triggering conditions" [14]. He says he has met the same pagination bug at three of his last four companies [1].
What to watch
- Whether the full post publishes before-and-after latency figures for a cursor migration on the same endpoint.
- A documented case of an agent loop breaking on shifting offsets, which the post so far asserts as a preference.
- Whether any mainstream ORM or API framework ships a cursor pager as its out-of-the-box default.