Explain the 'Oplog' (Operations Log) architecture. What happens if the Oplog is too small?

Question

Accepted Answer

The oplog (operations log) is the heart of MongoDB replication. It is a special capped collection named `oplog.rs` stored in the `local` database on every replica set member . When the primary performs a write operation, it records that operation as an entry in its oplog. Secondary nodes continuously poll the primary's oplog, copy these entries, and apply them to their own data sets. This asynchronous log-shipping mechanism keeps all secondary nodes synchronized with the primary while maintaining eventual consistency. // Switch to the local database
use local

// View oplog contents
db.oplog.rs.find().limit(5)

// Typical oplog document structure
{
  "ts" : Timestamp(1640995200, 1),  // Timestamp for ordering
  "t" : NumberLong(1),              // Term (for Raft protocol)
  "h" : NumberLong("123456789"),    // Unique operation ID
  "v" : 2,                           // Oplog version
  "op" : "i",                        // Operation type (i=insert, u=update, d=delete)
  "ns" : "mydb.users",               // Namespace (database.collection)
  "ui" : UUID("..."),                 // Collection UUID
  "o" : { _id: 1, name: "John" }     // Operation document
}

// Check oplog stats
db.oplog.rs.stats()

// View replication info
rs.printReplicationInfo()
// Shows oplog size, time range, and window length The oplog is a fixed-size, circular buffer . Each entry contains all the information needed to replay a write operation, including timestamp (`ts`), operation type (`op`), namespace (`ns`), and the actual document data . The timestamp field is critical for replication order and for calculating replication lag. The circular nature means that as new operations are added, the oldest entries are overwritten once the oplog reaches its maximum size. This size is configurable and determined during replica set initialization. An undersized oplog creates a severe operational risk. Since the oplog is a capped collection, it continuously overwrites the oldest entries with new ones. If a secondary node falls behind and the primary's oplog has overwritten the operations that secondary still needs to apply, that secondary enters a fatal state called 'too stale to catch up' . In this state, the secondary cannot continue replicating and must perform a full resync, which involves copying all data from another node—a time-consuming and resource-intensive process that can take hours for large datasets. Short oplog window: `rs.printReplicationInfo()` shows a window measured in hours or days; a healthy oplog should provide at least 24-72 hours of coverage Secondary nodes repeatedly entering RECOVERING state Error messages: 'could not find member to sync from' or 'replication error' in logs Increasing replication lag that never recovers Secondaries stuck with 'no source for replicated oplog entry' The optimal oplog size depends on your write workload. MongoDB's default oplog size varies by storage engine: for WiredTiger, the default is 5% of free disk space (up to 50GB) . General recommendations suggest sizing your oplog to accommodate at least 24-72 hours of peak write activity . This window allows time to recover from secondary outages, perform maintenance, or restore a node without requiring a full resync. For write-heavy workloads, consider 2-3 days of oplog coverage as a safety buffer. // For a standalone node (not recommended in production)
db.adminCommand({ replSetResizeOplog: 1, size: 163840 })  // Size in MB

// Better: Resize on a rolling basis across the replica set
// 1. Shut down the node
// 2. Restart as a standalone with a larger oplog size in config
// 3. Rejoin the replica set

// Or, in newer versions, dynamic resize is supported
db.adminCommand({
  replSetResizeOplog: 1,
  size: 50000,        // New size in MB
  minRetentionHours: 48 // Minimum oplog window in hours
})

Explain the 'Oplog' (Operations Log) architecture. What happens if the Oplog is too small?

The oplog is a capped collection that records all write operations in a replica set, enabling secondary nodes to replicate data changes. If the oplog is too small, secondaries may fall so far behind that they can no longer catch up and require a full resync.

Scenario Questions

Follow-up Questions