Mercari's TiDB gains came from storage knobs, while plan caching went back off

Mercari's Database Reliability Engineering team has published the second of three write-ups covering the TiDB work it did through the end of March 2026 ^[1]. The ranking is the useful part: the two changes that paid were storage-layer configuration, and the change most people would reach for first was reverted on nearly every cluster.

Raising `coprocessor.region-split-size` from 96MiB to 256MiB reduced cluster-wide region count by roughly 10% and improved TiKV CPU load by up to about 20% in one case ^[2][9]. Enabling the In Memory Engine took queries that had been taking 0.5 to 1.5 seconds down to around 0.1 seconds ^[3], a five- to fifteen-fold reduction ^[2]. Plan cache and instance plan cache are, at the time of writing, enabled on exactly one of their clusters and disabled on the rest ^[4].

The region mechanics explain why the first one worked. A region is the key-range unit by which TiKV manages and replicates data, and the unit of load balancing and failure recovery ^[5]. Fewer, bigger regions mean fewer RPCs for fetch paths like KvBatchGet and Coprocessor ^[6], at the cost of coarser load balancing and heavier work when regions have to move ^[7]. It is a cluster-global setting, so you cannot shrink regions for one table ^[8]. Mercari notes the gain only shows up when there is a lot of data and the workload fetches multiple rows in a batch; single-row reads or small datasets will not benefit ^[11]. Two operational details travel with it: the related region-merge thresholds need adjusting alongside ^[13], and on v8.5.0 through v8.5.2 an upgrade could silently flip the effective value to 256MiB, fixed in v8.5.3 as tikv#18503 ^[12]. Worth noting that 256MiB has been the default since v8.4.0 (v8.5 in the LTS line) ^[10], so Mercari was moving onto the upstream default, not away from it ^[1].

Plan caching failed for less obvious reasons. Mercari enabled it initially, then saw hit rates fall sharply when clusters went from 8.1 to 8.5, and turned instance plan cache back off after finding its overhead larger than its benefit ^[14][15]. Instance plan cache, added in 8.4, shares plans across a TiDB node rather than per session and is more memory efficient, but the sharing does not span the cluster and it sits on top of ordinary plan cache being enabled ^[16]. Non-prepared DML caching is off by default ^[17]. Two concrete causes: the non-prepared plan cache excludes queries that compare a numeric column against a string literal such as '123', and some Mercari application code embeds numbers as strings ^[18]; and the 8.5 optimizer change that added OR-condition pruning to predicate simplification is implicated in the post-upgrade drop ^[19].

The resource group finding is the one to copy immediately. Mercari uses resource groups to measure and cap per-tenant usage ^[20], and hit unexplained throttling where TiKV CPU spiked and the Task Wait QPS by Priority metric showed resource control holding queries back ^[21]. TiKV keeps per-priority queues and a CPU quota limiter that starves lower priorities under congestion, with quotas auto-adjusted from observed usage ^[22]; a bug in that auto-adjustment drove MEDIUM and LOW quotas to their minimum when no limiting was warranted, so MIDDLE-priority work waited while CPU sat idle ^[23]. Mercari was running default workloads at MIDDLE, isolated ones at LOW, and holding HIGH in reserve ^[24]. The fix in 8.5.5 disables auto-adjustment outright; the interim workaround was to set HIGH ^[25]. Their standing advice is to put OLTP traffic on Priority HIGH ^[26].

Watch for the third instalment, which should carry the In Memory Engine detail the summary only gestures at ^[1][3]. Also worth checking whether the 8.1-to-8.5 plan cache collapse Mercari attributes to OR-condition pruning ^[19] shows up in other people's upgrades, and whether anyone running below 256MiB regions sees the same 10%/20% split ^[2].