Build1 publisher3 min readPublished
Four eTRAKiT hosts running the same product cap a permit search at 100 to 1,000 rows
A dev.to write-up on the permit layer behind AddressIntel measures each host's row cap by hand, on one date, because a clipped eTRAKiT result set looks exactly like a complete one and the rows it drops are the newest.
The Engineer · Build desk

What happened
- As of 2026-09-16 the permit layer behind AddressIntel held 187,838 permits from 20 permitting jurisdictions, nineteen cities and towns plus unincorporated San Mateo County.
- Those jurisdictions publish through six vendor portal products, one municipal open-data feed, and one city whose only complete record is a monthly report posted as a document.
- One city's portal login was rejected on every run, and because the scraper printed a warning and returned an empty list, the pipeline reported success while the city vanished from the product for weeks.
- Every eTRAKiT host applies its result cap before sorting and emits no error when a search saturates.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost The cap has to be probed host by host, and a portal administrator can change it without notice, so the one-query-per-host measurement has to be repeated.
- exposure The cost of the silent gap landed outside the engineering team: a demolition-permit count of 0 for a covered city reached public copy written for builders choosing that market.
- constraint Any downstream filter keyed on issued date excludes 39,830 rows without saying so, among them 372 that the portals themselves mark Issued.
- decision Anyone adopting the round-number check is accepting false alarms on a jurisdiction that genuinely issued exactly 500 permits in a window. Covering hosts nobody can log into costs those false alarms.
The clip is server-side, and it lands before the sort. Rows come back in eTRAKiT's own internal order, which skews heavily oldest-first. A saturated query therefore loses the newest permits, and those are the only ones a recent-activity scrape exists to find [14].
Sorting does not get them back. A search for AT LEAST 01/01/2026 on the Hillsborough host, sorted by issued date descending, returns that query's 500 rows stopping at 2026-04-22, and the city has been issuing permits continuously since [18]. The sort only orders the rows that survived the cap [12].
So the check has to run before anything downstream trusts the rows, because the only signal the page gives is the size of the set [13]. Raising the grid's page size past the cap also removes the pager that would have read "page 1 of N" [12]. The two settings are independent: on the un-resized grid, N times the page size equals the cap exactly [17].
Those caps were measured on 2026-08-25, one query per host, page size raised to 2000 so the whole set arrived at once [15]. For the numbers to transfer to another eTRAKiT fleet, every host there would have to be configured the way these four were that day, and the write-up makes the opposite assumption: with a tenfold spread [28] across hosts of nominally the same product, the cap cannot be hardcoded once and forgotten [16]. Two hosts in the fleet are login-gated, so their caps are unmeasured, not absent, and the code keeps that distinction [19].
For those two, the test is whether a total lands exactly on a round number, on the reasoning that a real permit count hitting exactly 500 by coincidence is less likely than something having clipped it [20]. It is a likelihood argument, and it inherits two failure modes from its own logic. A jurisdiction that genuinely issued exactly 500 permits in the window trips it, and a host whose cap is not a round number slips past it [20].
One source accounts for most of the dataset's missing dates. Of the 39,830 rows with no issued date [6], 37,235 come from Palo Alto's open-data export, which publishes an application date and a lifecycle status for each permit, 33,641 of them Finaled, and no issue date [7]. That is 93% of the undated rows [24] and 70% of Palo Alto's own [25]. Take Palo Alto out and 2,595 of the remaining 134,678 rows lack an issued date, 1.9% [26], including 372 that the portals mark Issued [8].
Values are stored as the portal printed them, dates included. One Saratoga row carries an issue date of 2026-10-03, seventeen days after the write-up, against an application date of 2025-10-01 [9]. A permit issued seventeen days into the future is probably a keying error.
Storing values as the portal printed them is what makes the row counts readable at all. Palo Alto holds 53,160 rows and Foster City 28 [5], a ratio of about 1,900 to 1 [27], and the whole apparatus above exists to tell a quiet city from an under-collected one [5]. Downstream, 177,522 of the permits carry an assessor's parcel number and 128,324 carry a declared valuation above zero [4], so a valuation-weighted query runs on 68% of the corpus [23].
What to watch
- Whether the two login-gated eTRAKiT hosts ever get a credentialed probe. A probe would replace the round-number heuristic with a measured cap.
- Whether Palo Alto's open-data export starts publishing an issue date. An issue date would move 37,235 rows out of the undated bucket.
- Re-measurement of the four caps: a host reconfigured from 500 down to 100 would shrink collection until the next probe runs.