Cloudflare Logpush Transformers run a SQL query on each log batch before export

The cost argument depends on where the filter sits. Logpush applies the query before export ^[3], so a record the query drops is never delivered, and the destination never ingests or stores it. Cloudflare's stated case is that teams pay to export, ingest and store records they discard as soon as they arrive, and that filtering first can reduce downstream volume and cost ^[8]. The post's opening sections do not include a price for Transformers, a savings figure or a per-record overhead figure.

The 110 million figure is the rate of the whole delivery service, which runs across hundreds of Kubernetes pods ^[6]. Cloudflare says the wider pipeline ingests more than one billion events per second ^[7]. Those are fleet numbers. For them to predict what one customer's job costs, per-record overhead would have to stay flat whether a query only filters or runs a regex on every record.

Cloudflare says it embedded a SQL transformation engine in the export pipeline while keeping performance and resource use predictable ^[13]. A section heading names Apache DataFusion and Arrow as the execution layer ^[14]. The text we have ends before it explains how.

The redaction in the worked example is a regex. It rewrites the last two dotted groups of ClientIP, so 203.0.113.42 becomes 203.0.XXX.XXX ^[9]. The pattern needs a literal dot before each of two digit groups at the end of the string. An IPv6 address has colons and no dots, so as written the pattern would not match one, and regexp_replace would return it untouched ^[11]. The sample record carries an IPv4 address ^[9]. Customers can preview a Transformer before attaching it ^[5], so an IPv6 record in the preview is a cheap test.

The host filter has a similar edge. It is a literal list of two names ^[10], and a host not on the list is not excluded ^[12]. Production estates do acquire hostnames. Each edit saves a new version ^[5], so there is a history of when the list changed.