Interval width does not scale with rows. It scales with the root of how many separate examples you actually hold, so a thirtyfold overstatement of that count tightens every error bar by about 5.5, according to the dev.to post [7]. Do the division: one over the root of thirty is 0.183, so the interval you print covers roughly 18 percent of the width the evidence supports [1]. The post rounds that to a fifth [8], which is the same statement.
The penalty is a property of the window, not of the model, which makes it portable arithmetic. Swap in a seven-day label on the same 3,300 rows and you get about 471 effective examples, with intervals near 38 percent of warranted width [2]. Move to the 365-day label and 2,940 rows stand in for nine observations, an overstatement of 327 times that compresses intervals by roughly 18, to about five percent of their honest width [3]. The post's verdict on an estimate like that is that it carries no real information [17].
Row count over window length is a crude estimator, and the author says so while arguing it beats nothing [14]. It is optimistic in one specific way, because it treats any two observations more than one window apart as independent draws. Across nine years of a real series the data generating process has usually changed more than once [13], so those nine annual observations are not nine draws from one distribution, and the effective count for a model that assumes stationarity is lower than the division implies.
A test suite will not catch this one. The data is valid and the split is temporally correct and the code is right; the defect lives in an assumption underneath the metric [11]. The post's line for it is that these assumptions can fail silently, without raising an exception [11]. So this belongs in review, next to the metrics table, rather than in CI.
The remediation is cheap at the bottom. Report effective sample size in the same table as the metrics, so nobody reads one without the other [14]. Keep training on every correlated row, because they still help the fit, but derive intervals, error bars and significance tests from a non-overlapping subset [15]. Above that sits blocked cross-validation with purging and embargo: contiguous segments stay together, purging removes the examples whose windows straddle a fold boundary, and an embargo gap stops the edge from sharing information at all [16]. Financial ML has used that for years, and per the post everywhere else that windows appear has underused it [16].
In my context the first rung buys the most. One extra column makes the assumption visible, and a visible assumption is one somebody can argue with at review.
