An audit of 4,959 robot episodes argues for a data gate before you blame the policy

An open-source tool called RDA (Robot Data Audit) was run across 12 LeRobot-format datasets covering 4,959 episodes, and its author's stated takeaway is that a large fraction of those episodes carried signals worth investigating before being fed into training ^[1][2][14]. That matters mostly because the check is cheap: the core audit runs locally and does not require a GPU ^[3], which makes it a plausible gate to put upstream of the more expensive habit of rewriting the policy architecture first.

The premise is worth stating plainly, because pipelines usually stop one step short of it. A robot dataset can have valid Parquet files, correct schemas and complete metadata while still containing excessive idle motion, action discontinuities, sampling problems, distribution anomalies, or episodes that deserve human review ^[4]. "Can I load the dataset?" is necessary and not sufficient ^[5].

RDA checks 13 metrics across three layers ^[6]. The deterministic layer covers missing frames, NaN and Inf values, schema consistency, timestamp validity and joint limits, and hard structural failures can produce an EXCLUDE verdict ^[7][8]. The statistical layer covers sensor synchronization, sampling jitter, velocity and acceleration anomalies, action discontinuity and temporal sufficiency, which the author treats as signals for investigation rather than proof an episode is unusable ^[9][10]. The third layer is contextual: idle ratio, distribution characteristics and state-space coverage ^[11].

The design decision that will matter to anyone wiring this into CI is the split between the workflow verdict (PASS, REVIEW, EXCLUDE) and the evidence level (HARD_FAIL, RISK_SIGNAL, UNVERIFIABLE) ^[12]. The author's reason is to stop an audit tool from converting weak statistical evidence into confident labels ^[13]. High action discontinuity is a RISK_SIGNAL, not a finding that the demonstration is corrupted ^[10][13].

The idle numbers are the loudest result. Median idle ratio across the tested datasets ranged from 20.8 percent to 93.3 percent, and ten of the twelve datasets had median idle ratios above 65 percent ^[15][16], which is about 83 percent of the sample ^[17]. If most frames show the robot doing very little, what the policy learns from them is an open question that depends on task and architecture ^[18].

The counterexample in the same post is the useful part. Two datasets from the same xArm platform but different tasks diverged sharply: one had a 20.8 percent median idle ratio and 767 of 800 episodes rated PASS, the other 83.3 percent median idle and 562 of 800 rated REVIEW ^[19][20]. That is 95.9 percent PASS against 70.3 percent REVIEW ^[21][22] on identical hardware, and it is why the author calls blanket rules such as "delete every episode above 70 percent idle" dangerous ^[23]. A high idle ratio may just be how the task is performed ^[24].

Two honesty notes, both from the author. There was no independent human ground truth for the benchmark, so the counts are not a confusion matrix and cannot support precision or recall claims ^[25]. And the post does not publish the overall PASS/REVIEW/EXCLUDE totals across all 4,959 episodes ^[14], which is the number anyone sizing a review queue would want.

Watch for that breakdown, and for a filtered-versus-unfiltered training comparison. Until someone reports a policy-performance delta from gating on these metrics, the case for RDA rests on cost: it is local, it is deterministic in its first layer, and it tells you where to look ^[3][7].