Replaying 8,900 real files cut a file-type detector's false alarms from 105 to zero

The harness is one loop: for each file it records the detector's verdict on the file as it sits, then copies the file to a temp folder with the extension removed and asks the core inspector which extension it would guess ^[7]. The original extension is the answer key, and the copy arm only runs on files of 12 MB or smaller ^[22]. For an app written to catch invoice.pdf.exe, the extension is an odd thing to grade against ^[24]. It is also the only label available without hand-labelling.

That tension is visible in the fixes. Every false-alarm cause the post lists is a case where the extension was not telling the truth about the bytes: .rs that is not Rust, .msg that is not Outlook, .CSS that is HTML, .so holding a Windows DLL ^[14]. The change went into the dictionary ^[14]. The headline accuracy number is agreement with extension convention, allowing the aliases the harness defines, and the post calls that a stricter test than identifying the format ^[11].

A Windows box has roughly 9,000 .js files and 8,000 .dll files, so an uncapped walk measures JavaScript ^[21]. Sampling decides what the number means. The harness caps each extension per root, 30 per extension under System32 to depth 3, and caps every extension at 90 across all 35 roots ^[20][19]. Against roughly 8,000 .dll files, 90 is a bit over one percent of them ^[4]. The sweep is 8,900 files over about 600 extensions, an average near fifteen each ^[8][5].

Read the gains as error rates: on extensions the app knows, 69.9% to 90.1% means error fell from 30.1% to 9.9%, so about two thirds of the errors are gone ^[9][1]. On the roughly 130 extensions the post says ordinary users meet, 76.5% to 93.0% is 23.5% error down to 7.0% ^[10][2]. Unidentified files fell from 5,408 to 3,533, which against the 8,900-file sweep is about 61% down to about 40% ^[12][3].

One Makefile feature, `(\w[\w.\-/]*\s*)*$`, backtracked exponentially across lines and stalled the detector on a file called Appraiser_Data.ini ^[15]. That hang is the part I would copy first. Timing every file then turned up four more rules that go quadratic on a single whitespace-free line, all of them starting with `\w+` ^[16]. The post flags two shapes to grep for, nested quantifiers with a `\s` that eats newlines, and an unanchored `\w+` at the start ^[17].

The cheapest fix was the confidence rule. The author wrote that "high confidence" requires two different features to match, and that "Repeating one feature, however many times, never counts as certainty" ^[18].

For the 90.1% to say anything about your build, your inputs would have to be written by the same population of tools: Office, Git, the Windows Kits, a Python install, one developer's dev folder ^[20]. The loop ports to any format-sniffing code with a directory to walk. The app is closed source and ships on the Microsoft Store, so the run cannot be reproduced from a repository ^[23].