A 32 GB collection with a four-hour restore is a scheduling problem, not a scaling one

Whoz, a SaaS vendor whose platform tracks consultant time as records it calls worklogs, let one MongoDB collection grow to 530 million documents and just over 32 GB [s1c1][s1c2][s1c3]. At that size a backup took nearly an hour and a restore up to four, and the team wrote that those windows had become operationally risky [s1c4][s1c5][s1c6].

Note what is absent from that description. The company says its core issue was not query throughput: worklogs from three years ago are rarely read, and when they are, performance expectations are low [s1c7]. The problem was operational overhead, specifically backup time, restore time, and the cost of running large batch operations across the whole dataset [s1c8].

The volume figure is worth sitting with. Just over 32 GB across 530 million documents works out to roughly 60 bytes of stored data per document ^[1]. That is a small dataset by any storage measure and a large one by document count, which is why the pain shows up in operations that iterate rather than in operations that seek. The restore number is the load-bearing one: at up to four hours, any recovery time objective shorter than four hours was already unachievable, whether or not anyone had written it down ^[2]. Restore also ran about four times longer than backup, so the number that matters is the one nobody rehearses ^[3].

The third symptom is the most telling and the least numeric. Whoz had not dared run a schema migration at full scale, and said so plainly, calling that a warning sign in itself [s1c9]. A maintenance operation you decline to attempt is not a deferred task, it is a capability you no longer have.

That framing decided the architecture. Sharding is MongoDB's native horizontal scaling mechanism, distributing a collection across shards that each need their own replica set, typically three nodes for high availability [s1c10][s1c11]. Whoz ruled it out on the grounds that it would mean paying the full cost of a distributed system to solve what was essentially a data locality problem [s1c12]. The team then looked at offloading WiredTiger data files to S3-compatible object storage, checked the documentation and asked MongoDB support, and was told the configuration is not supported: the engine has internal tiering in some contexts, but MongoDB does not expose it for general use [s1c13][s1c14].

What remained was an application-level split: historical worklogs into a separate collection, recent ones left in place, both exposed transparently to the application [s1c15]. Within that, one collection per year was rejected for complexity, since cross-year queries need unions, new years need routing changes, and migrations become multi-collection work [s1c16]. Whoz notes that MongoDB views do support $unionWith but that chaining it across five or six collections starts to feel fragile [s1c17]. The sliding-window variant, one recent and one older collection with a moving boundary, was described as clean in theory but requiring a periodic re-partitioning process that has to be automated and monitored [s1c18].

Two things to watch. The account as supplied breaks off while weighing that sliding window, so the design Whoz shipped and what it cost to backfill are not in evidence [s1c19]. And the growth rate is the variable that governs everything here: the company reports the collection growing faster each year, driven both by new clients and by users splitting activity into finer-grained entries [s1c20]. On your own systems, the equivalent check is cheap: pull the trend line on backup duration, and time a restore rather than assuming one.