Build1 publisher3 min readPublished
Unlinking the -shm file under a live connection caused corruption that recurred twenty hours later
SQLite's own list of corruption causes is mostly a list of things processes do to the file. On one agent's state database the cause was two writers on one directory, and the repair removed the coordination those writers needed.
The Engineer · Build desk

What happened
- PRAGMA integrity_check stops reporting after 100 problems, quick_check omits the table-versus-index comparison and can miss damage, and foreign_key_check only reports foreign key violations.
- A ptrmap line in the output means autovacuum's pointer map disagrees with the pages, a stray, short or duplicated write that the post attributes in practice to two writers on one file.
- The post offers no in-place repair: stop every writer, copy the .db with its -wal and -shm, pipe .recover into a fresh file, then validate that file before swapping it in.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Anyone running a gateway, a CLI and a second server against one state directory now has to collapse that to a single writer or funnel writes through one process, because the post classes the alternative as an application bug to fix in the application.
- cost Recovery costs rows. .recover rebuilds only what surviving pages allow and dumps unattributable rows into lost_and_found, leaving someone to reconcile the agent's history by hand.
- exposure A host running with synchronous=OFF, or a disk that acknowledges writes it has not flushed, accumulates damage that surfaces only on a later read, and the agent's stored history is what is exposed.
The -shm file is the wal-index, the shared memory SQLite uses to coordinate readers and writers [11]. Unlink it while a connection still has the database open and that coordination is gone [11]. The author of the dev.to post did exactly that, re-ran a repair, and got the same table back with the same error about twenty hours later [2][3]. The second writer was still running: the agent had two processes pointed at one state directory [26]. Which message you got decides what you do next. SQLITE_CORRUPT (11) is page-level damage inside a file that is still recognisably SQLite, so there are surviving pages to recover [4]. SQLITE_NOTADB (26) means the header is not SQLite at all, which the post puts down to the wrong path, a zero-length file, or something else writing over it [5]. The author also recommends reading the first error in a stack trace rather than the last, because by the last one the real cause is usually several layers down [6]. If integrity_check output mentions ptrmap, autovacuum's pointer map disagrees with what is actually on the pages, and the post calls that structural damage from a stray, short or duplicated write, which in practice usually means two writers on one file [10]. It is not an index logic bug, and REINDEX will not clear it [10]. The ranking the post offers is one operator's frequency order from agent and desktop app trackers, not a measured population [16]. For it to transfer, your deployment has to look like those: single-file state on a machine a person uses, several hand-run processes able to open it, and a state directory a user can move. A fleet with one long-lived writer and no sync client has a different first entry. All five ranked causes are things done to the file, its location, or its durability settings [27]. Deleting or renaming the sidecars under a live connection [11]. Putting the state file on NFS, SMB, OneDrive, Dropbox or iCloud, where SQLite's advisory locking assumes a real local filesystem and a sync client rewrites pages underneath it [12]. Two processes writing one file without agreeing on locking, which the post calls an application bug and not a SQLite one [13]. Copying a live database with a file copy instead of the backup API or VACUUM INTO [14]. And PRAGMA synchronous=OFF, or a disk that reports a write as synced when it is not [15]. Of SQLite's own list at sqlite.org/howtocorrupt.html [17], the author wrote: "Nearly everything on it is something we do to the database, not something the database does to itself" [18]. SQLite's acknowledged defects are the elimination criteria. The post names two, both narrow and old: a WAL race writing to a WAL-mode database at section 8.1, and corruption after switching between rollback and WAL mode with a VACUUM in between at 8.4 [19]. Local disk, one writer, no sidecar fiddling and a sane busy_timeout puts you in that territory, and the author says collect a repro instead of re-reading your own code [20]. Recovery starts with stopping every writer, including the gateway and the CLI. "If a process still holds the file, everything below is theatre," the author wrote [22]. Copy state.db with its -wal and -shm to scratch, because the .db alone drops committed data that lives only in the WAL and can make a healthy database look corrupt [23]. Then pipe .recover into a new file: .dump stops at the first sign of corruption, while .recover reassembles what the surviving pages allow and parks rows it cannot attribute in a lost_and_found table [24]. Validate the new file with integrity_check and foreign_key_check, REINDEX, and keep the old file [25]. The available text of the post breaks off mid-sentence at the last step, counting the rows lost [28].
What to watch
- Whether agent frameworks default their state directory to a user folder that OneDrive, Dropbox or iCloud syncs.
- Whether anyone files a reproducible case against SQLite's WAL race (8.1) or the rollback-to-WAL-with-VACUUM path (8.4) from an agent runtime.
- Whether agent runtimes start enforcing a single writer per state file instead of letting a gateway, a CLI and a second server open it.