Changing only the name on a CV moved its rank across three million model comparisons

A hiring model learns to predict who recruiters advanced, not who performs. The dev.to piece describes the system's job as finding the statistical signature of the kind of person a company hires, then applying it to new applicants ^[1]. Train on that target and the best achievable model reproduces the past decision. Deviation from the historical pattern registers as error during training. Where the historical record is skewed by gender, race, age or class, the model is trained to match that skew ^[2].

For the 85% figure to transfer to a screener you are evaluating, several things have to hold. Kyra Wilson and Aylin Caliskan tested leading language models ranking real CVs against real listings ^[6]. If a vendor instead fine-tunes on your own hire history, you get a different function with the same failure mode, because the target is still a record of past human choices ^[1]. If the tool is a rules-based knockout on years of experience, the University of Washington numbers do not cover it. The study measured pairwise preference between candidates, not who was hired ^[7].

554 CVs against more than 500 listings gives at least 277,000 CV-listing pairs, and over three million comparisons works out to roughly eleven per pair ^[13]. Most of that variation comes from swapping names onto the same documents. The count of distinct CVs is 554 ^[7].

Removing the name is the first control anyone reaches for, and the study reports that it does not hold. The models inferred a candidate's likely identity from the schools listed, the cities lived in, dates that hint at age, and the words an applicant used to describe their own work ^[10]. Amazon hit the same wall six years earlier. Reuters reported in 2018 that its experimental CV scorer, trained on ten years of applications to a heavily male engineering workforce, taught itself that men were preferable ^[3]. The tool downgraded CVs containing the word "women's" and penalised graduates of at least two all-women's colleges ^[4]. Amazon edited out those specific signals, then dropped the project because it could not be confident the system would not find other proxies for the same bias ^[5].

The scale argument in the dev.to piece is reasoning about mechanism: a learned pattern gets applied to far more people than any single biased human could reach, and "the same" is what anti-discrimination law was written to interrupt ^[11]. In my view that part holds. A recruiter's prejudice varies by day, by mood and by candidate. A deployed model applies the same weights to every applicant in the queue until someone changes them.

The piece does not cite an enforcement action or a court finding against a deployed screener, so the legal exposure here is inferred from disparity rates. The rates themselves are specific. Against an even split, 85% for white-associated names sits 35 points above parity and 11% for female-associated names sits 39 points below ^[14]. Black men did worst, with the models preferring other candidates in close to every test ^[9].