BuildNot yet confirmed elsewhere1 publisher2 min readPublished
ZOrder with date as a secondary key ran 78% slower than partitioning in a Delta Lake benchmark
A Delta Lake benchmark on 50 million records found ZOrder 78% slower than date partitioning on time ranges when date was a secondary key. Its author makes Liquid Clustering the default for diverse or evolving query patterns on the strength of a single benchmark.
The Engineer · Build desk
Three figures from the author's Delta Lake benchmark: on stable filter columns, Liquid Clustering nearly five times slower than ZOrder while scanning ten times more data; a fifty-million-record dataset; four workload types benchmarked.
| Figure | Value | Note | Claim |
|---|---|---|---|
| Liquid Clustering, stable filters | 5 x slower, nearly | while scanning ten times more data | 4 |
| Records in benchmark dataset | 50,000,000 records | 6 | |
| Workload types benchmarked | 4 types | 1 |
What happened
- Across the four workload types, Liquid Clustering posted the lowest latency on point lookups, time-range aggregations and large aggregations.
- Where a query predicate matches a stable set of filter columns, ZOrder pruned close to optimally and Liquid Clustering ran nearly five times slower while scanning ten times more data.
- On point lookups Liquid Clustering had the lowest latency even though it scanned more raw data than ZOrder, and the author calls bytes scanned a misleading proxy.
- The test compared partitioned, ZOrdered and Liquid Clustered layouts using real Databricks query profiles.
Why it matters
- decision Teams choosing a layout are told to run their top three query patterns against partitioned, ZOrdered and Liquid Clustered copies of a representative sample, since one author's dataset cannot settle it.
- cost The author counts the elimination of manual OPTIMIZE jobs and periodic full table rewrites as an operating saving that comes with Liquid Clustering.
- constraint The author says most published comparisons come from Databricks itself or reason from first principles, so this run has few independent measurements to be checked against.
The author's explanation for the date penalty is that ZOrder disperses sequential date values across files [3]. Date partitioning gets range-scan efficiency for free, and the dispersal destroys it [3]. For any team that ZOrdered on several columns with date somewhere after the first key, the author wrote, time-range queries are almost certainly taking the hit [14].
The 78% transfers only to tables that resemble the test. Date has to sit in the second or third slot of the ZOrder expression [2]. Date partitioning has to be a live alternative, since that is the baseline the figure is measured against [2]. And the data has to behave like the test dataset [6].
The evidence is one author's benchmark. The author says the questions came from production Delta Lake work at TCS UK (civil aerospace) and IBM (Shell Maritime) [11]. The available text ends mid-sentence, before any results table, and does not report absolute latencies, cluster size or Delta version.
The author's account of the multicolumn loss is that Liquid Clustering's broader locality optimization trades deterministic pruning for workload flexibility, and that the trade becomes a liability when queries are narrow and predictable [10]. ZOrder won only on stable, highly selective multicolumn predicates where the filter columns match the clustering key [5].
The author sorts teams with one question: do the top queries consistently filter on the same two or three columns? If yes, ZOrder those columns. If not, choose Liquid Clustering [8]. Everything outside that case falls to the default the author sets, Liquid Clustering for diverse or evolving query patterns [15].
We think the test is sound. "Stable" is the hard word in it, because it is a forecast about queries nobody has written yet.
What to watch
- Whether anyone reproduces the 78% penalty with date in the second or third ZOrder slot on a different dataset.
- Whether a run on a stable multicolumn filter workload matches the nearly fivefold gap the author reports for Liquid Clustering.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence40
- Adoption
- Insufficient
- Hype gap+30
- Incentives
- Insufficient
- Confidence45
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Across four workload types benchmarked on Delta Lake, Liquid Clustering delivered the lowest latency for point lookups, time-range aggregations and large aggregations.
- [2]
When a date column sits as a secondary or tertiary key in a ZOrder clustering expression, time-range queries perform significantly worse than on a simple date-partitioned table, seventy-eight percent slower in this benchmark.
- [3]
Sequential date values are dispersed across files, destroying the range-scan efficiency that partitioning provides for free.
- [4]
When a query predicate precisely matches a stable set of filter columns, ZOrder's file pruning is close to optimal and Liquid Clustering can be nearly five times slower while scanning ten times more data.
- [5]
ZOrdering performed better only for stable, highly selective multicolumn predicate workloads where the filter columns precisely match the clustering key.
- [6]
The benchmark compared partitioned, ZOrdered and Liquid Clustered layouts across four analytical workload types on a fifty-million-record dataset using real Databricks query profiles.
- [7]
Liquid Clustering delivers the lowest latency on point lookups despite scanning more raw data than ZOrder; the author calls bytes scanned a misleading performance proxy.
- [8]
The author's diagnostic question is whether top queries consistently filter on the same two or three columns; if yes, ZOrder those columns, otherwise choose Liquid Clustering.
- [9]
The author recommends running the top three query patterns against all three configurations, partitioned, ZOrdered and Liquid Clustered, on a representative sample of your own data.
- [10]
Liquid Clustering's broader locality optimization trades deterministic pruning for workload flexibility, and that trade-off becomes a liability exactly when queries are narrow and predictable.
- [11]
The questions behind the benchmark were shaped by the author's production challenges architecting Delta Lake platforms at TCS UK (civil aerospace) and IBM (Shell Maritime).
- [12]
Liquid Clustering eliminates the operational burden of manual OPTIMIZE jobs and periodic full table rewrites.
- [13]
The author says most published comparisons either come from Databricks itself or reason about the trade-offs from first principles without measuring actual query behavior.
- [14]
If your team ZOrdered on multiple columns and date is not the first key, this is almost certainly affecting your time-range query performance.
- [15]
Liquid Clustering should be the default for any team whose query patterns are diverse or evolving, and outperforms ZOrder on most workloads.
Sources
1 independent publisher whose own reporting we read for this story.
- infoq.comArticle: When Should You Use Liquid Clustering? Benchmarking Lakehouse Layout Strategies
1 article · October 12, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.