Markup filled 263,968 of a scraped product page's 267,361 tokens

The count works by subtraction. Each page was fetched once with requests.get and a Chrome User-Agent, then one category of markup was removed at a time and the remainder re-counted with tiktoken's o200k_base, each category measured after the ones above it so the rows sum to the fetched total ^[8]. On the Next.js docs page the largest single line was hydration JSON, most of it inside 65 self.next_f.push calls that repeat the page content as escaped strings ^[9]. You pay for the prose, then you pay for the escaped copy of the prose. The Hacker News front page has almost no scripts or styles and still came in at 90% markup, because it is built from layout tables ^[10].

Subtract the visible text from that product page total and 263,968 tokens of markup remain, about 98.7% of what was fetched ^[1][2]. At the $2.00 per million input rate the post prices against, reading it once costs roughly 53 cents, and the part anyone wanted costs under a cent ^[3]. Across all ten pages the totals were $3.04 as HTML and $0.25 as Markdown ^[12], a pooled ratio near 12 to 1 ^[6].

The refusal message the post prints is "Request too large ... Limit 30000, Requested 57266" ^[16]. The request was about 1.9 times the account limit, and the visible text on that page would have used 2% of it ^[4].

Latency tracked size. On gpt-5.6-terra the raw version took 2.0 to 2.8 times longer to first token on the four pages above 130K tokens, and 1.0 to 1.4 times longer on the four below 75K; two other models ran up to 5.7 times slower to start ^[15]. Four raw pages loaded into 396K-token prompts still returned the right answer in all three runs ^[18].

Conversion is a subtraction too. Markdown drops style blocks, SVG paths, class and data- attributes and scripts, and keeps headings, lists, tables and link URLs ^[19]; the Hacker News front page came back as tables carrying story titles, URLs, scores and comment counts in 3,644 tokens instead of 11,817, a cut of about 3.2 times ^[20][5]. JSON-LD sits inside a script block, so it leaves with the scripts. Asked for a product's price, rating and review count, the model got all three from the HTML and only the price from the Markdown ^[21]. That same Markdown retained 99.4% of the page's words, because the rating and review count are 2 tokens out of 16,844 ^[23]. Decodo's recommended fix is to read the application/ld+json blocks with an HTML parser, since quoting varies, and send the model the prose ^[24].

Every figure is the author's own runs from September 2026 ^[7], published by Decodo, whose Web Scraping API takes a markdown flag that runs the conversion server-side ^[5][25]. Ten pages, one unrendered fetch each ^[8]. Across OpenAI, Anthropic and Google tokenizers the same HTML measured up to 50% larger while the pooled ratio held between 11 and 14 times, and the post's own advice is to trust the ratio and re-count the dollars for your model ^[14][27]. It also says docs, news and storefronts would each need a different rule ^[11].