How would you decide the initial number of shards for a new collection when the eventual data size is uncertain?

The shard count is set at collection creation and cannot be changed without re-sharding, which is a heavier operation than adding replicas or moving shards. So the initial count must be planned generously for the eventual size, not just the current size. The rule of thumb is to size for 2-3 years of growth, or for the upper bound of the size estimate, and to over-provision rather than under-provision. If the eventual data size is uncertain, choose a shard count that is comfortable for the high end of the estimate: for a collection that might reach 500M points, 16 shards is a reasonable starting point (each shard holds ~30M points, which fits comfortably on a node). If the collection might reach 50M points, 4-8 shards is sufficient. The cost of over-provisioning is a small increase in fan-out overhead and coordination cost; the cost of under-provisioning is a re-sharding operation, which requires migrating data, may cause downtime, and is operationally complex. Over-provisioning is almost always the right choice when the eventual size is uncertain.

The mechanism behind this rule is that the shard count determines the parallelism of the collection and the granularity of data placement, but it also determines the fan-out cost of every query. More shards means more parallel work but more coordination and merge overhead. The sweet spot depends on the data size and the query pattern. For a small collection, a single shard is fine; for a large collection, many shards are necessary to distribute the data and the query load. The uncertainty is about the eventual size, so the decision should be based on the upper bound. There is a second consideration: custom sharding. If the collection uses custom sharding by tenant, the number of shards is the number of shard keys (or a multiple), and the initial count should accommodate the eventual tenant count. This is different from hash sharding, where the shard count is a global parameter. The third consideration is the replication factor, which multiplies the storage and compute requirements and should be planned separately from the shard count. Adding a replica is cheap compared to re-sharding, so the replication factor can be adjusted later.