Build1 publisher3 min readPublished
USAspending's hasNext flag goes false at record 10,000 with 3.28 million contracts matched
The federal award search endpoint needs no API key and exactly one filter. Its paging flag reports the end at record 10,000, and a directly requested page 150 still returns a full 100 rows of the same descending sort.
The Engineer · Build desk

What happened
- USAspending.gov serves federal award data over a public REST API: no key header, and no signup or quota form either. You POST JSON to it and get JSON back.
- In testing, page_metadata.hasNext turned false at record 10,000 whatever the page size, while the page immediately before that boundary still reported true.
- Pages requested beyond the cap still returned full result sets, with the descending Award Amount sort continuing cleanly from page 100 into pages 101, 102 and 150.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- exposure Any collector already written around hasNext has been stopping at 10,000 rows and reporting success, so the downstream table is short by design and the logs show nothing wrong.
- decision Query design has to move before paging: slices by NAICS code or date window must each match under 10,000 awards, and the reference code puts that check in front of the loop.
- cost Each query now costs an extra count request, and a full sweep of the FY2026 contract set at the maximum page size runs to 32,784 requests.
- constraint Because the data has to be gathered in slices, the merge needs a stable key, and the reference loop uses generated_internal_id while flagging Award ID as the wrong one.
The boundary tracks the record offset, not the page number. Three combinations landed on it: limit=100 with page=100, limit=50 with page=200, and limit=10 with page=1000 all returned hasNext:false, while pages 99, 199 and 999 respectively returned true [10].
The rows keep coming after the flag says stop. Page 100 ended at an Award Amount of 115,774,074.02, page 101 opened at 115,773,474.42 and closed at 114,636,312, and page 102 opened at 114,622,680.94 [11]. Page 150 returned a full 100 rows opening at 78,246,081 and reported hasNext:false like the rest [11]. At limit 100, page 150 covers records 14,901 through 15,000, so 5,000 rows past the cap came back to a plain request [18].
With the unbounded contracts filter, the count endpoint returned 3,278,345 contracts for FY2026 to date, measured 2026-07-20 [8]. The 10,000 records reachable through the paging flag are 0.31 percent of that [16]. Covering the whole set at limit 100 would be 32,784 pages [17]. A loop that trusts hasNext returns HTTP 200 at every step and stops at exactly 10,000 with no error and no warning, so the only symptom is a dataset that ends on a suspiciously round number [12].
So the working order of operations is:
1. POST the same filters object to `spending_by_award_count` first. A `fields` key there is accepted and ignored, so reusing the search body does no harm [9]. 2. If the count is above 10,000, split the query. The reference code logs "partition this query further" at exactly that threshold [15]. 3. Page `spending_by_award` and break on an empty `results` array, not on hasNext [13].
`filters.award_type_codes` is the only required filter, and omitting it returns 422 with `Missing value: 'filters|award_type_codes' is a required field`; `time_period`, which plenty of sample code sends, is not required [4]. `limit` runs from 1 to 100, and sending 101 returns `Field 'limit' value '101' is above max '100'` [5]. `sort` must be a member of your `fields` array or the call returns 400 naming the mismatch [6]. A GET returns 405 with `{"detail":"Method \"GET\" not allowed."}` [2]. Both the slashed and unslashed paths returned 200 with no redirect in testing [3].
When you merge the partitioned slices, the reference loop dedupes on `generated_internal_id` and marks Award ID as the wrong choice [14]. The search response does not include a total; its top-level keys are `spending_level`, `limit`, `results`, `page_metadata` and `messages`, and `page_metadata` holds only page, hasNext, last_record_unique_id and last_record_sort_value [7]. A single response cannot tell you whether you got everything. The count call is the only check available [13].
What would have to be true for the 10,000 ceiling to be irrelevant to your job: every query you run matches fewer than 10,000 awards. The FY2026 contract total is three orders of magnitude past that [8]. A top-100-by-amount dashboard is fine. A bulk extract runs into the cap.
What to watch
- Whether USAspending starts returning an error at record 10,000 instead of hasNext:false, or raises the offset cap.
- Whether directly addressed pages past 10,000 records keep returning full rows, since the workaround depends on behaviour the response reports as finished.
- Whether the count endpoint keeps accepting and ignoring the fields key, which is what lets one request body serve both calls.