Skip to content

Build1 publisher3 min readPublished

Four eTRAKiT hosts running the same product cap a permit search at 100 to 1,000 rows

A dev.to write-up on the permit layer behind AddressIntel measures each host's row cap by hand, on one date, because a clipped eTRAKiT result set looks exactly like a complete one and the rows it drops are the newest.

The Engineer · Build desk

Illustration accompanying Four eTRAKiT hosts running the same product cap a permit search at 100 to 1,000 rows

What happened

  • As of 2026-09-16 the permit layer behind AddressIntel held 187,838 permits from 20 permitting jurisdictions, nineteen cities and towns plus unincorporated San Mateo County.
  • Those jurisdictions publish through six vendor portal products, one municipal open-data feed, and one city whose only complete record is a monthly report posted as a document.
  • One city's portal login was rejected on every run, and because the scraper printed a warning and returned an empty list, the pipeline reported success while the city vanished from the product for weeks.
  • Every eTRAKiT host applies its result cap before sorting and emits no error when a search saturates.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • cost The cap has to be probed host by host, and a portal administrator can change it without notice, so the one-query-per-host measurement has to be repeated.
  • exposure The cost of the silent gap landed outside the engineering team: a demolition-permit count of 0 for a covered city reached public copy written for builders choosing that market.
  • constraint Any downstream filter keyed on issued date excludes 39,830 rows without saying so, among them 372 that the portals themselves mark Issued.
  • decision Anyone adopting the round-number check is accepting false alarms on a jurisdiction that genuinely issued exactly 500 permits in a window. Covering hosts nobody can log into costs those false alarms.

The clip is server-side, and it lands before the sort. Rows come back in eTRAKiT's own internal order, which skews heavily oldest-first. A saturated query therefore loses the newest permits, and those are the only ones a recent-activity scrape exists to find [14].

Sorting does not get them back. A search for AT LEAST 01/01/2026 on the Hillsborough host, sorted by issued date descending, returns that query's 500 rows stopping at 2026-04-22, and the city has been issuing permits continuously since [18]. The sort only orders the rows that survived the cap [12].

So the check has to run before anything downstream trusts the rows, because the only signal the page gives is the size of the set [13]. Raising the grid's page size past the cap also removes the pager that would have read "page 1 of N" [12]. The two settings are independent: on the un-resized grid, N times the page size equals the cap exactly [17].

Those caps were measured on 2026-08-25, one query per host, page size raised to 2000 so the whole set arrived at once [15]. For the numbers to transfer to another eTRAKiT fleet, every host there would have to be configured the way these four were that day, and the write-up makes the opposite assumption: with a tenfold spread [28] across hosts of nominally the same product, the cap cannot be hardcoded once and forgotten [16]. Two hosts in the fleet are login-gated, so their caps are unmeasured, not absent, and the code keeps that distinction [19].

For those two, the test is whether a total lands exactly on a round number, on the reasoning that a real permit count hitting exactly 500 by coincidence is less likely than something having clipped it [20]. It is a likelihood argument, and it inherits two failure modes from its own logic. A jurisdiction that genuinely issued exactly 500 permits in the window trips it, and a host whose cap is not a round number slips past it [20].

One source accounts for most of the dataset's missing dates. Of the 39,830 rows with no issued date [6], 37,235 come from Palo Alto's open-data export, which publishes an application date and a lifecycle status for each permit, 33,641 of them Finaled, and no issue date [7]. That is 93% of the undated rows [24] and 70% of Palo Alto's own [25]. Take Palo Alto out and 2,595 of the remaining 134,678 rows lack an issued date, 1.9% [26], including 372 that the portals mark Issued [8].

Values are stored as the portal printed them, dates included. One Saratoga row carries an issue date of 2026-10-03, seventeen days after the write-up, against an application date of 2025-10-01 [9]. A permit issued seventeen days into the future is probably a keying error.

Storing values as the portal printed them is what makes the row counts readable at all. Palo Alto holds 53,160 rows and Foster City 28 [5], a ratio of about 1,900 to 1 [27], and the whole apparatus above exists to tell a quiet city from an under-collected one [5]. Downstream, 177,522 of the permits carry an assessor's parcel number and 128,324 carry a declared valuation above zero [4], so a valuation-weighted query runs on 68% of the corpus [23].

What to watch

  • Whether the two login-gated eTRAKiT hosts ever get a credentialed probe. A probe would replace the round-number heuristic with a measured cap.
  • Whether Palo Alto's open-data export starts publishing an issue date. An issue date would move 37,235 rows out of the undated bucket.
  • Re-measurement of the four caps: a host reconfigured from 500 down to 100 would shrink collection until the next probe runs.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories