Build1 publisher2 min readPublished
A cached strptime only wins when the same date shows up 2,740 times
A week of measurements on one Mac mini puts the lru_cache wrapper around strptime at 3x when dates repeat and slightly slower when they do not. A Polars 2.0 release candidate finishes the same report in 0.13 s.
The Engineer · Build desk
What happened
- Every figure comes from one machine, a Mac mini M4 Pro running Python 3.14.6, with five runs per configuration and medians reported.
- On a million rows carrying only 365 distinct dates, the lru_cache version of the report finished in 1.29 s as a whole process and 1.03 s inside the script.
- The write-up puts the cache trick at about 3x and only when inputs repeat; on a million unique dates it ran 6 percent slower than doing nothing.
- All three versions wrote output files byte-identical to the original script's, all 48 rows.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Choosing between the wrapper and the rewrite depends on a repeat rate you can state for your own data, because the same four lines of code are a 3x on one input distribution and a small tax on another.
- constraint The 25x is an in-script comparison, so a job that starts a fresh interpreter on a schedule collects the whole-process ratio instead, which is 9.9x on the repeated-date data.
- cost The 0.13 s costs a dependency and a rebuild of the report around read_csv, str.to_datetime, group_by and sort, paid once by whoever maintains the script.
- exposure Anyone adopting on these figures is adopting a pre-release, and the numbers have to be taken again when 2.0 final ships.
lru_cache hashes the argument tuple and checks a dict before it calls the function. On a hit, a stored datetime comes back and strptime never runs. On a miss you pay the hash, the lookup, the call and the store. Spread 365 distinct dates over a million rows and each string arrives about 2,740 times, so 99.96 percent of lookups hit [1]. With a million unique dates every lookup misses, and the wrapper costs 3.24 s inside the script against 3.06 s plain [7][6].
The 3x and the 25x come from different columns of the same table. Inside the script the cache is 3.0x; measured as whole processes, start to exit, it is 2.6x [2]. Polars against the cached version is 25.8x in-script and 9.9x as a process [3]. A script on an hourly schedule pays the process number, because it pays interpreter startup on every run. The release candidate's two in-script medians, 0.04 s and 0.03 s, differ by 0.01 s, the last digit the table reports [5][8].
Subtract the columns and the plain script spends 0.26 s outside its timed section; the Polars script spends 0.09 s [6]. The note does not explain why the process that imports Polars carries less startup than the one that does not.
For the rewrite's number to transfer, one format string has to cover the column. The parse is `pl.col("date").str.to_datetime("%Y-%m-%d %H:%M:%S")` [13]. Mixed formats or nulls in the same column take a different code path. The rest of the report also has to fall out of a group_by and a sort, and in this one it does [12].
Two medians for the plain script are now published against the same data: 3.71 s in last week's video and 3.36 s this week, 10 percent apart [2][3][5]. The penalty the cache imposes on unique inputs, 5.7 percent at the process level, is smaller than the gap between those two baselines [4].
The write-up draws its line by how often the script runs. "If the script is yours and small, the cache is the cheaper fix," it says. "If it runs every hour on real data, the rewrite pays for itself the first day." [14]
The same note re-ran its uv benchmark on version 0.12.11 and landed within 4 percent of 0.11.6 across the same eight packages, so that upgrade buys features and not speed [16]. The Polars figures come from 2.0.0-rc.1, and the post says to re-check when 2.0 final ships [15].
What to watch
- Whether the 2.0 final release holds the rc.1 timings on the same 48-row report.
- A re-run on Linux with a slower disk, since every figure here comes from one Mac mini M4 Pro.
- A version of the test where the date column needs more than one format string.