Object or file storage: the wrong pick stays quiet until inode exhaustion

The useful part of this comparison is not the requirement lists, it is when each mistake becomes visible. Put a database on object storage and the error reports itself early: the dev.to writeup by Ethan Carter puts object latency at millisecond level with no in-place mutation ^[6], against the sub-millisecond random reads, in-place byte updates, ordered fsync and lock files that PostgreSQL, MySQL, MongoDB and SQLite expect from a filesystem or block device ^[7]. Two of those four properties are precisely the ones object storage does not have ^[20]. Put user photos on an ext4 volume instead and nothing reports anything at all until inodes run out ^[1]. One error costs you a slow afternoon in development. The other costs an afternoon two years later, on a volume somebody else provisioned.

The two anecdotes in the post are the same workload an order of magnitude apart. The filesystem falls over around 10 million files ^[21]; the bucket holding the same photos reached 100 million ^[22], a 10x gap on identical data ^[23]. And the object path got there without a code change ^[10], which tells you where the migration work lives. It is not on the side that scaled.

So the test runs against the caller, not the data. The operating system opens, reads, writes and seeks; it does not issue HTTP PUTs, and Carter's advice is not to fight that ^[8]. His own shorthand is that random writes and file locks mean file storage, while HTTP PUTs and billions of objects mean object storage ^[9]. Uploads, data lake Parquet, backups and ML checkpoints all sit on the object side of his list ^[4], and none of them call seek(). Shared design files over SMB, build trees over NFS and enterprise home directories sit on the other side because they need directory browsing, file-level permissions and application transparency ^[16].

He also says most mature stacks run both side by side ^[5], which makes this a per-path decision rather than a per-company one. The lakehouse exception fits that reading: Iceberg and Delta are named as the niche where things do work over object storage ^[6], and that is a table format built for object semantics, not a database talked into them.

The hedge is a mount, and the writeup is careful about it. You can get filesystem semantics over S3-style storage, Carter says, but the mount layer will lie to you in small ways ^[13]. RustFS, whose documented feature table covers S3 core, versioning, bucket replication, event notifications and bitrot protection, ships no FUSE driver at all ^[14]. If your plan for the POSIX requirement you have not fully specified yet is "we will mount it", that plan depends on a component the server you picked may not offer.

This is one practitioner's checklist rather than a benchmark, and the only latency figures on offer are "sub-millisecond" and "millisecond-level" ^[3][6]. It is still enough, because the answer does not turn on how many milliseconds. It turns on whether anything in the path mutates bytes in place.