Build1 publisher3 min readPublished
FoxyInvoice's Angular site gave crawlers one word until a build-time prerender filled it in
FoxyInvoice prerenders every public Angular route to static HTML at build time so crawlers that skip JavaScript get the full page text. A build step and a one-line drift check take the place of the always-on SSR server its team declined to run.
The Engineer · Build desk
What happened
- Fetching FoxyInvoice's Angular marketing page with curl returned a title tag reading "Invoicing" and an empty app-root element, nothing else.
- A try_files rule at the edge proxy serves the prerendered static files to any client, and Angular swaps in the live app once it boots in a browser.
- FoxyInvoice publishes an llms.txt markdown file at its site root listing the product, each template page with a one-line description, and contact points.
- Every deploy POSTs all sitemap URLs to the IndexNow endpoint, and the first ping returned HTTP 202 with the pages in Bing's queue that evening.
- The edge now logs every request as rotated JSON, and one script sorts the traffic by user-agent, path and crawler class.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost Teams adopting the pattern take on a prerender script and a drift check that must stay in step with the sitemap on every build.
- decision Under a default-deny robots.txt, each new public page needs a deliberate edit before any crawler may read it, so a forgotten entry hides a page from AI and search crawlers alike.
- exposure If the post's account of AI crawlers holds, any SPA that ships an empty shell offers GPTBot, ClaudeBot and PerplexityBot a title and nothing to quote.
Server-side rendering was the obvious fix, and chapter 11 of the FoxyInvoice build series on dev.to turns it down [18]. An always-on SSR server for pages the author puts at 95% app shell would have broken what the team calls its boringness constraint [4]. Instead, a post-build script writes a static copy of every public route when the SPA compiles, with no new server and no SSR framework [5].
Each copy has its own title and meta description, a canonical URL, Open Graph tags, JSON-LD, and the page's full body copy: line items, how-to steps, FAQ [5]. The author gives the reason for the last item: "meta tags describe content; answer engines quote content" [16]. I agree with putting the body copy first. The post says GPTBot, ClaudeBot and PerplexityBot do not run JavaScript [1], so for them the prerendered HTML is the whole page. Googlebot, in the same post, is "tolerant, eventually" [17].
The part I would copy first is the guard. Prerender pipelines tend to fail quietly, with a route that exists in the app but never gets a static copy. FoxyInvoice's build fails when the sitemap and the template data drift apart, and that one-line count check caught a real bug on its first run [6].
The serving path, as described, does not branch on user agent, so a crawler and a person get the same first response [7]. For the design to transfer, a team's public pages have to be knowable when the build runs. A site whose public pages change per request would need frequent rebuilds, or the SSR server FoxyInvoice chose not to run. FoxyInvoice's public pages are template pages, and the build already holds their data [6].
The robots.txt is an allowlist. It opens with `User-agent: *`, allows the root, /pricing, /privacy, /terms and /templates, then closes with `Disallow: /` [8]. With a single stanza for every agent, the AI crawlers the post names get the same list as search crawlers [19]. Private pages also get `X-Robots-Tag: noindex, nofollow` from the edge proxy because, the author wrote, "robots.txt disallow alone never guarantees de-indexing" [9].
The evidence so far is about crawling and indexing. Within days of the JSON logging going live, the team saw search crawlers reading robots.txt and template pages, and stray /wp-admin scans being absorbed [14]. The post does not report a fetch by GPTBot, ClaudeBot or PerplexityBot, or a FoxyInvoice page cited in an AI answer. The link from IndexNow to those engines rests on the author's statement that the Bing-powered endpoint feeds several of them [12]. For the indexing and impressions view that logs cannot give, the team registered with Google Search Console and Bing Webmaster [15].
What to watch
- JSON access logs showing GPTBot, ClaudeBot or PerplexityBot fetching the prerendered template pages or llms.txt would test the chapter's premise directly.
- A FoxyInvoice template page cited in an AI answer engine's response would show whether the Bing IndexNow route reaches those engines as the author says.
- Search Console and Bing Webmaster impression data in a later chapter would show whether the same-evening Bing queue turned into indexed pages.