Are Bulk Operations in MongoDB atomic?

Question

Accepted Answer

Bulk operations in MongoDB are a way to perform multiple write operations (like inserts, updates, or deletes) in a single database command, which can significantly improve network efficiency. However, contrary to a common misconception, a bulk operation is not an atomic transaction. If your bulk operation contains writes to multiple documents, it does not have the 'all-or-nothing' property. Some writes can succeed while others fail, and these partial results are committed to the database . It's important to understand that MongoDB's atomicity guarantee still applies at the **single-document level**. This means that within a bulk operation, each individual `insertOne`, `updateOne`, or `deleteOne` operation is atomic for the document it affects . For example, if you have a bulk operation that updates Document A and then inserts Document B, the update to Document A is an atomic operation. If it succeeds, the change to Document A is persisted, even if the subsequent insertion of Document B fails . Consider this example using a Node.js driver. The bulk operation aims to insert two new documents. If the second document causes a unique index violation, the result shows that the first insert still succeeded.

```javascript
const { MongoClient } = require('mongodb');

async function run() {
  const client = new MongoClient('mongodb://localhost:27017');
  await client.connect();
  const db = client.db('testDB');
  const collection = db.collection('testCollection');

// Ensure a unique index on the 'sku' field
  await collection.createIndex({ 'sku': 1 }, { unique: true });

// Prepare a bulk operation with two inserts
  const operations = [
    { insertOne: { 'document': { 'sku': 'A', 'name': 'Product A' } } },
    { insertOne: { 'document': { 'sku': 'A', 'name': 'Duplicate SKU' } } } // This will fail
  ];

try {
    const result = await collection.bulkWrite(operations);
    console.log('Bulk operation result:', result);
  } catch (error) {
    console.error('Bulk operation caught an error:', error.message);
    // Check how many operations succeeded
    console.log('Partial result (successful ops):', error.result);
  }

// Query the collection to see the state of data
  const docs = await collection.find({}).toArray();
  console.log('Documents in collection:', docs);
  await client.close();
}

run();
```

**Expected Output (conceptual):**
```json
Bulk operation caught an error: E11000 duplicate key error
Partial result (successful ops): { insertedCount: 1, ... }
Documents in collection: [
  { "_id": ObjectId(...), "sku": "A", "name": "Product A" }
]
``` **Ordered Operations (Default)**: When you execute an ordered bulk operation, MongoDB processes the operations in the sequence you provided. If an error occurs during the processing of one of the write operations, MongoDB returns without processing any remaining operations in the list . In this case, the point of failure is clear, but all operations before the error are persisted. **Unordered Operations**: With an unordered operation, MongoDB can execute the operations in parallel to maximize throughput. If an error occurs during the processing of a write operation, MongoDB will continue to process the remaining write operations in the list . The final result will contain information about which operations failed and which succeeded. **Key Takeaway**: In both ordered and unordered modes, writes that succeed before an error are committed to the database and are **not rolled back**. For use cases that require true atomicity across multiple documents (an 'all-or-nothing' guarantee), MongoDB has supported **multi-document ACID transactions** since version 4.0 . By wrapping your bulk operation logic within a transaction, you ensure that if any operation fails, the entire set of changes is rolled back. However, it's worth noting that transactions come with a performance overhead. The MongoDB documentation suggests that a large majority (80-90%) of applications using the document model can achieve data consistency without them by properly structuring their data and leveraging the atomicity of single-document operations . For your use case of scraping and upserting 1 million products, an unordered `bulkWrite` with upserts remains the most efficient approach, as the occasional failure of a single product upsert likely does not necessitate rolling back the entire batch.

Are Bulk Operations in MongoDB atomic?

Bulk operations in MongoDB are not atomic across multiple documents, meaning individual write operations within the batch are atomic, but the batch as a whole is not

Are Bulk Operations in MongoDB atomic?

Bulk operations in MongoDB are not atomic across multiple documents, meaning individual write operations within the batch are atomic, but the batch as a whole is not

Scenario Questions

Follow-up Questions