Hot shard and noisy neighbor; dedicate a shard or collection to the outsized tenant
The problems fall into three categories: shard imbalance, resource contention, and operational impact. Shard imbalance: with custom sharding, each tenant is placed on a shard, and a tenant with 100x the data of others will make its shard much larger than the rest. That shard will have more segments, more memory usage, longer index builds, and potentially longer query latency. Resource contention: the large tenant's queries and writes consume more CPU and I/O, which can affect the other tenants on the same node - the classic noisy neighbor problem. Operational impact: the large tenant's data will dominate backups, snapshots, and reindexing, and a problem with that tenant's data (a corrupted segment, a bad batch import) can affect the shared collection's availability. The fix is to isolate the large tenant: either by giving it a dedicated shard (so its data is on its own shard and its queries do not compete with others on the same shard), or by giving it a dedicated collection (so it has its own segments, indexes, and optimizer state, and can be tuned independently).
The mechanism that makes isolation effective is that Qdrant's resource consumption scales with the number of segments and the size of the index. A shard with 100x the data has 100x the segments (or larger segments), which means more memory for the HNSW graph and payload indexes, more CPU for traversal, and more I/O for disk-based access. By giving the large tenant a dedicated shard or collection, you bound its resource consumption to its own shard and prevent it from affecting others. A dedicated collection also lets you tune the large tenant's configuration independently - higher m for better recall, more replicas for availability, a different quantization scheme - which is not possible in a shared collection where the configuration is collection-wide. The decision between a dedicated shard and a dedicated collection depends on whether the large tenant needs different configuration. If it does, a dedicated collection is the right choice. If it needs the same configuration but more capacity, a dedicated shard (or several shards) within the shared collection is sufficient. In both cases, the tenant's data must be migrated, which requires a plan for moving the data without downtime.
Shard imbalance: the large tenant's shard is much bigger, with more segments and higher resource use.
Noisy neighbor: the large tenant's queries and writes consume CPU and I/O that affect other tenants.
Operational impact: backups, snapshots, and reindexing are dominated by the large tenant.
Dedicated shard: the large tenant gets its own shard within the shared collection.
Dedicated collection: the large tenant gets its own collection, with independent configuration.
Migration: moving the tenant's data requires a plan for zero-downtime cutover.
Tiering: establish thresholds (data size, query rate) that trigger isolation.
Monitoring: track per-tenant resource usage to detect an outsized tenant early.
The trade-off is between isolation and operational complexity. A dedicated collection gives the large tenant the best performance and the most flexibility, but it adds a collection to manage, monitor, and back up. A dedicated shard within the shared collection is simpler but does not allow independent configuration. The common mistake is to leave the large tenant in the shared collection and hope it does not cause problems, which eventually leads to a noisy-neighbor incident. The second mistake is to give the large tenant a dedicated collection without a migration plan, which causes downtime. The third mistake is to not have thresholds for isolation, so the decision is made reactively during an incident. The fourth mistake is to not monitor per-tenant resource usage, so the problem is not detected until it affects other tenants. The fifth mistake is to assume that the large tenant is the only one - if the platform grows, more tenants will become large, and the isolation strategy must scale. Version note: custom sharding and the is_tenant flag have evolved across Qdrant releases, and the ability to move a tenant between shards or collections may vary. If you are designing for tiered multitenancy, verify the version's support for shard-level placement and migration.
Version-dependent: custom sharding, the is_tenant flag, and the ability to create shard keys with specific shard counts are recent additions to Qdrant and have evolved. If you are on an older version, the tiering options may be more limited, and the practical approach may be to create a separate collection for the large tenant from the start. Verify the version's support for tiered multitenancy before designing the strategy.
A new enterprise tenant signs up with 100x the data of your typical tenant. Explain the problems this causes in a shared collection and what you would do.
A teammate says the tenant filter handles isolation. Explain why isolation is not enough when the tenant is much larger than the others.
Your shared collection has a hot shard because one tenant is much larger. Describe the diagnosis and the remediation.
You need to move the large tenant to a dedicated collection without downtime. Describe the migration plan.
Design a tiered multitenancy strategy with thresholds for when a tenant moves from shared to dedicated shard to dedicated collection.
You have multiple large tenants and they are all growing. Describe the capacity planning and the isolation strategy.
You are designing a multi-tenant platform where tenant sizes follow a heavy-tailed distribution. Describe the architecture, the isolation strategy, and the capacity planning.
Derive the optimal isolation threshold as a function of tenant size, query rate, and the cost of a dedicated collection.