Skip to content

Build1 publisher3 min readPublished

Anti-bot systems now score the session, which means your proxy pool is not a mitigation

A dev.to post argues that Akamai, Cloudflare, DataDome and HUMAN grade behaviour across a whole session. If it is right, the line item to grow is session capacity, not IP volume.

The Engineer · Build desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

Illustration accompanying Anti-bot systems now score the session, which means your proxy pool is not a mitigation
Generated illustration

What happened

  • The post asserts that rotating proxies solves a problem from about five years ago, and that teams whose response to a new block is 'add more IPs to the pool' have not had that fix work cleanly in a while.
  • The older model of bot detection was mostly about the request itself: whether the IP is on a known list, whether the user agent is on a known list, and whether the address has made too many requests too fast.
  • The newer generation of anti-bot systems, the ones behind Akamai, Cloudflare, DataDome, PerimeterX/HUMAN and similar platforms, evaluate the pattern of behaviour across an entire session rather than a single request in isolation.
  • Session continuity is one behavioural signal: a real visitor arrives and takes a sequence of actions that build on each other, cookies persist, a session token carries across pages, and the same client keeps showing up in a way that looks like one visitor rather than a new stranger every time.
  • Browser environment consistency is another signal: a genuine browser has a stable, internally consistent set of characteristics (rendering behaviour, available APIs, hardware-reported details) that stay the same request to request, whereas a scraping setup that swaps configuration on every attempt looks like a different device on every request, which is itself an anomaly.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

A post published on dev.to under the PromptCloud services account makes a claim worth taking seriously by anyone who runs a crawler: rotating proxies solves a problem from roughly five years ago [16][1]. The consequence, if the description is accurate, is budgetary rather than technical, because a team that answers every new block by buying more IPs is funding the wrong resource [1][8].

The old detection model, according to the post, looked at the request in isolation: is this IP on a list, is this user agent on a list, has this address made too many requests too fast [2]. The newer systems behind Akamai, Cloudflare, DataDome, PerimeterX/HUMAN and similar platforms are described as evaluating the pattern of behaviour across an entire session instead [3]. Four named vendors is a small sample of the market, but it covers a lot of the front doors a commercial crawler knocks on [17].

The signals listed are all things a proxy cannot supply. Session continuity: cookies persist, a session token carries across pages, and the same client keeps reappearing in a way that reads as one visitor rather than a new stranger each time [4]. Browser environment consistency: a genuine browser has stable, internally consistent rendering behaviour, available APIs and hardware-reported details, so a setup that swaps configuration on every attempt presents as a different device on every request, which is itself the anomaly [5]. Pacing: human interaction has natural variability and pauses, so a fixed interval, or an interval no person could sustain, stands out [6]. The post argues these are increasingly combined over a session's lifetime into something closer to a running confidence score than a pass/fail check [7].

That is the part that inverts the economics. The one-request-per-IP, rotate-on-every-attempt, no-session pattern was built for a world where the address was the primary signal, and against a behavioural system that exact pattern is what gets caught [8]. A client that appears once, carries no history, shows a slightly different fingerprint than the last visitor and repeats every few seconds looks like one automated process cycling addresses, not many humans [9]. The architecture built for the old problem manufactures the evidence for the new one [10].

The remedies described are unglamorous and they cost throughput. Treat the session, not the request, as the unit of work, persisting for a realistic span with its own cookies and state [12]. Freeze the client environment for the life of that session: whatever browser engine, headers and configuration it starts with should not vary per request [13]. Pace to realistic usage rather than maximum extraction, which the post notes is also easier on the source's infrastructure [14]. Use a real rendering engine where the content actually requires it, since much of what gets called evasion is just executing JavaScript the way a browser does [15]. The post explicitly warns against the alternative of countering each fingerprinting technique as it appears, calling that a losing, shifting game and a poor foundation for production [11].

Two things to watch. First, whether your own throughput per identity is now bounded by session lifetime and pacing rather than pool size, because that is where the capacity planning moves [18]. Second, the provenance: this is a single vendor-adjacent post with no measurements attached [16]. The signal list is checkable against your own block rates, and that is the test worth running before the next proxy invoice.

Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories