WiredTiger manages concurrency during massive updateMany operations through document-level locking, MVCC with in-memory update chains, and a ticket-based system that caps concurrent write transactions, but performance can degrade due to lock yields, unique index overhead, and memory pressure from large update chains.
WiredTiger, MongoDB's default storage engine, employs a sophisticated concurrency control mechanism to handle massive updateMany operations. Unlike older storage engines that used process-wide locking, WiredTiger provides document-level concurrency using Multi-Version Concurrency Control (MVCC) and optimistic locking . During a large updateMany, WiredTiger must balance throughput, consistency, and resource constraints while potentially affecting thousands of documents.
WiredTiger uses document-level locks rather than collection or database locks, allowing multiple clients to modify different documents in the same collection simultaneously . For most write operations, WiredTiger employs optimistic concurrency control. When two operations conflict on the same document, one will experience a write conflict, causing MongoDB to transparently retry the operation . This approach maximizes throughput while maintaining consistency.
When an updateMany modifies documents, WiredTiger does not immediately write changes to disk. Instead, it creates new WT_UPDATE structures and prepends them to in-memory update chains associated with each document . These chains enable MVCC by allowing concurrent readers to see their own consistent snapshot while writers modify the document. Readers traverse the chain from newest to oldest until finding a version visible to their transaction snapshot . This design ensures that readers don't block writers and vice versa .
Ticket System: WiredTiger caps concurrent write transactions using a ticket system, defaulting to a maximum of 128 write tickets . During massive
updateManyoperations, write tickets can become exhausted, causing queueing and increased latency.Yielding (Cooperative Scheduling): Long-running
updateManyoperations periodically yield (release locks) to allow other operations to execute. The highnumYieldsvalue incurrentOpoutput indicates frequent yielding, which prevents starvation but can extend operation duration .Intent Locking: WiredTiger acquires intent locks at the global, database, and collection levels even with document-level concurrency . The
currentOpexample shows global, database, and collection locks held in write mode during the entire multi-update operation .Flow Control: In high-write scenarios, MongoDB may activate flow control, throttling writes to keep secondary replication lag within limits. The
flowControlStatsincurrentOpshow time spent waiting for flow control .
A critical concurrency consideration for massive updateMany operations is the behavior of update chains. WiredTiger has an internal threshold (approximately 1000 updates) for the length of an update chain on a single document . When this threshold is exceeded, WiredTiger performs a copy-on-write, rewriting the entire document to a new memory location. During a massive updateMany that repeatedly updates the same documents, this can cause significant memory pressure, increased disk I/O, and degraded performance . This is particularly problematic for large documents with frequently updated arrays, where update chains can grow rapidly.
When updateMany modifies a field with a unique index, concurrency handling becomes significantly more complex. WiredTiger must check uniqueness constraints for each document, which can introduce additional locking and validation overhead. Historical data shows that updates on unique indexes were up to 60x slower than on non-unique indexes in some versions, though recent optimizations have narrowed this gap . The overhead is particularly pronounced when updating values to keys that sort before the original value, requiring more expensive index traversal .
Concurrent massive updates create significant memory pressure. When readers materialize the latest version of a document by traversing long update chains, they allocate memory outside the tracked WiredTiger cache. This untracked memory can grow unbounded with the number of concurrent readers, potentially causing memory spikes and instability . Additionally, when the history store accumulates many versions (due to pinned timestamps), locality decreases, forcing more I/O to retrieve old versions .
Batch Processing: Split massive
updateManyoperations into smaller batches to reduce lock holding times and yield more frequently, allowing other operations to interleave.Document Structure Redesign: For frequently updated arrays, consider sharding large documents into parent-child structures to prevent update chain bloat .
Index Review: Evaluate whether unique indexes are necessary on fields updated by massive operations, as they add significant concurrency overhead .
Cache Sizing: Ensure WiredTiger cache is appropriately sized (default is 50% of RAM minus 1GB) to accommodate update chains and concurrent reader materialization .
Monitor Yields: Track
numYieldsincurrentOpoutput to detect excessive yielding that may indicate concurrency contention .