Eight months of JSON as a database: the bill came due on slug identity, not read latency

The mechanism here has little to do with file systems. A slug assembled from course, semester, subject name and year is a primary key derived from content, and the author's university voids the assumption that made it look unique: one subject is issued under a different code per branch, so KCS078 for computer science, KDS078 for data science and KOT076 as an open elective all reduce to the same string ^[5]. The three PDFs had different hashes and different byte counts, and one address between them ^[7].

In a relational store that key gets declared once and the second insert fails while the schema is still fresh in someone's memory. That is the author's own reading of what the flat file cost him: with no constraint to violate, the duplicate row stays perfectly valid and permanently unreachable ^[9].

The proportions are worth writing down. 147 records were in a slug collision, which is 8.7 percent of the file ^[2], and the unreachable subset comes to 5.6 percent ^[3]. Across 1,699 records and 699 distinct subject codes, that is about 2.4 papers per code ^[6], so the namespace was always tight enough for this to happen.

The second half is the part that travels. The fix builds an index of which record owns each base slug and hands the rest a code-qualified URL ^[10], but the 147 colliding records were two different populations: 95 with genuinely distinct subject codes, and 52 duplicate uploads and placeholder rows whose IDs end in -dummy, sharing the same code as the record they collided with ^[12]. For the second group the code suffix disambiguates nothing, and because the matcher still honours a legacy code-first URL for backwards compatibility, the original record answers the new URL, wins the lookup, and 301s the visitor back where they started ^[13].

Note the detection ratio. The bad deploy presented as 0.06 percent of the swept surface ^[4]: fifty-two broken records, one visible symptom ^[7]. Nothing inside the application reports it, because the sections render and the pages load; the only party keeping score is Google, collecting 52 URLs that bounce ^[14].

There is also a number the post does not reconcile. The sweep checked 1,615 URLs against 1,699 records on disk, leaving 84 unaccounted for ^[5]. The sweep covers the sitemap, not the dataset, so the audit that found this class of bug is not the audit that proves the remaining records are addressable. Meanwhile the read path, the thing every argument about flat files is about, never featured: the file is under a megabyte, the workload is append-mostly and read-heavy, and a full parse per request has never woken him up ^[4].

His conclusion, that a full sweep is non-optional after any URL-shaped change and that the belief is not specific to flat files ^[15], holds. Any application computing slugs in code carries the same failure mode. Postgres only helps to the degree that the derived key is actually declared to it as unique.