How does the MongoDB Node.js driver handle a bulk operation that exceeds the 16MB BSON limit?

The driver's size calculation is precise. When building a batch, it tracks both the currentBatchSize (number of operations) and currentBatchSizeBytes (total BSON size). For each new operation, it calculates maxKeySize + bsonSize and checks if this would push the batch over the limit . The driver specifically sets ignoreUndefined: false during BSON size calculation to ensure accurate sizing, even when documents contain undefined values that might otherwise be omitted from size estimates . This prevents the under-counting that could otherwise cause batches to exceed the 16MB limit at the server level.

This automatic splitting behavior was not always present in all MongoDB tools. In older versions of the mongo shell (prior to 3.2), bulk operations that exceeded 16MB would simply fail with an error like 'Object size 16795903 exceeds limit of 16793600 bytes' . This was fixed in a Jira ticket (SERVER-23107) that improved the shell to split by size, bringing it in line with the behavior of the Python and C drivers. The Node.js driver has long included this intelligent batching behavior, making it robust for large-scale data operations . Modern versions of the driver (3.6+) continue to handle this seamlessly .

While the driver handles splitting automatically, you can optimize performance by being mindful of your operation sizes. For documents approaching the 16MB limit, even a single operation may require its own batch, which can impact throughput. For optimal performance with large datasets, consider batching 1,000-5,000 operations per bulk write call, as this gives the driver good opportunities for parallelization without creating excessive overhead . Also remember that unordered operations (ordered: false) allow MongoDB to execute batches in parallel, which can significantly improve throughput for independent operations .