The YAML file is the reconciliation script now

A write-up published on dev.to under the handle shell_qa lays out the full module layout of a layered, configuration-driven data validation framework in Python, in which test cases are YAML files rather than hand-written reconciliation scripts ^[1][3]. It matters because the defects it targets, missing rows and subtle rounding errors, are precisely the ones that survive manual verification and land in production ^[2][4].

The shape is unremarkable, which is the point. According to the post, run_regression.py reads config/execution.yaml for group-level on/off switches, then walks test case files named TC_{NNN}_{TYPE}.yaml; validation_runner.py routes each to a count, data, recon, or file validator, and df_loader.py pulls both sides into DataFrames via csv_loader.py or sql_executor.py ^[5][6]. Each YAML declares a tc_id that must match the filename, an enabled flag, a type, a human-readable description that feeds the summary report, and source and target blocks pointing at paths ^[6][7]. Queries live in sql/src/ and sql/tgt/ as real .sql files instead of inline strings, so a diff shows what changed ^[8].

The sequencing advice is the part worth stealing. Run the count check first: a count failure tells you the load or the pipeline broke before you spend compute on cell-level comparison ^[9]. The framework also requires tolerance to be set explicitly in YAML, with the stated warning not to inflate it to hide a real discrepancy absent documented business justification ^[10]. That is the honest admission in the design. Tolerance is the field that converts a rounding-error failure into a green run, and putting it in version-controlled config at least makes widening it an auditable act rather than a quiet edit inside someone's script.

Most of the remaining guidance is about not lying to yourself. Explicit ORDER BY clauses, because DataFrame comparison assumes deterministic row order, and no SELECT * in production tests, because schema drift then fails silently ^[11]. Source and target queries must return matching column names and types, or a shape mismatch masks the actual defect ^[12]. Where names differ, an explicit column_map block beats trusting positional alignment ^[13]. Credentials come from environment files with pooled connectors in dedicated modules ^[14], and real production data never enters the repository; masked or anonymized sets go under data/src/ and data/tgt/ ^[15].

Two operational details will bite teams. Control is gated twice, at group level in execution.yaml and per case through the enabled flag, and both must be true for a case to run, so a test that quietly did not execute has two hiding places ^[16][1]. Tests also run alphabetically by filename, which makes the numbering scheme a scheduling decision rather than bookkeeping ^[17]. Skipped cases are meant to stay on disk as enabled: false rather than be deleted, preserving history and audit trail ^[18]; execution calls sit inside try/except so one malformed query does not take the suite down ^[19]. Each cycle writes report/TC_000_SUMMARY.html with branding embedded as base64 ^[20], caps displayed mismatches at roughly the top 1,000 rows, and offers a full CSV download for root-cause work ^[21]. Logs rotate to logs/framework.log ^[22].

What to watch. First, whether tolerance values creep upward through the git history of a migration, because that history is now the record of how much error the business agreed to accept ^[8][10]. Second, whether QA actually authors the YAML, or engineers keep writing it and the ownership shift stays theoretical ^[3]. Third, note the limits of the material: this is a blueprint and a set of conventions, with no reported figures on defects caught, runtime, or migration outcomes ^[2].