How do TTL (Time-To-Live) indexes work and what are their limitations?

Question

Accepted Answer

Time-To-Live (TTL) indexes are special single-field indexes in MongoDB that enable automatic data expiration. When you create a TTL index on a date field, MongoDB automatically removes documents after a specified number of seconds . This feature is widely used for managing temporary data such as user sessions, event logs, and cached data that only needs to persist for a finite period . ```javascript
// Basic TTL index that expires documents 3600 seconds (1 hour) after the indexed date
db.eventlog.createIndex(
  { "createdAt": 1 },
  { expireAfterSeconds: 3600 }
)

// If you have an array of dates, MongoDB uses the earliest date in the array
db.sessions.createIndex(
  { "loginDates": 1 },
  { expireAfterSeconds: 86400 } // 24 hours
)

// For precise expiration at a specific date/time
db.tasks.createIndex(
  { "expireAt": 1 },
  { expireAfterSeconds: 0 } // Expires exactly at the specified date
)
```

When the indexed field contains an array, MongoDB uses the earliest (minimum) date value in the array to calculate the expiration threshold . If `expireAfterSeconds` is set to 0, documents expire exactly at the datetime specified in the indexed field. MongoDB implements TTL expiration through a dedicated background thread called the `TTLMonitor`. This thread wakes up approximately every 60 seconds, scans all TTL indexes across the instance, and removes expired documents . The deletion process runs only on the primary node of a replica set; secondary nodes replicate these delete operations through the oplog . You can monitor TTL activity using the `serverStatus().metrics.ttl` command, which shows the total number of deleted documents and TTL passes . **Non-Immediate Deletion**: TTL indexes do not guarantee that documents will be deleted immediately upon expiration. The background thread runs every 60 seconds, creating a potential delay. Additionally, heavy system load may delay the deletion further . **Single-Field Only**: TTL indexes must be single-field indexes. Compound indexes cannot include the `expireAfterSeconds` option . You cannot create a TTL index on a field that already has a non-TTL single-field index . **No Capped Collection Support (Historically)**: Traditionally, TTL indexes could not be created on capped collections. However, MongoDB 7.1+ now allows TTL indexes on capped collections for specific use cases . **Performance Impact**: TTL deletion operations consume CPU, I/O, and cache resources. In high-write scenarios, data insertion may outpace TTL deletion, causing storage growth. The single-threaded nature of TTL processing can create performance bottlenecks . **Date Field Requirements**: The indexed field must contain BSON Date type values. If the field contains non-date values or is missing entirely, the document will never expire . **ExpireAfterSeconds Modification**: You cannot modify `expireAfterSeconds` using `createIndex()`. To change this value, you must either use the `collMod` command or drop and recreate the index . **Storage Fragmentation**: Frequent deletions can lead to storage fragmentation. MongoDB mitigates this by enabling `usePowerOf2Sizes` for collections with TTL indexes, which allocates more space to reduce fragmentation . Production experience reveals significant challenges with TTL indexes at scale. A single `TTLMonitor` thread processes all TTL indexes sequentially, meaning multiple collections with TTL indexes cannot be cleaned in parallel . This can lead to data accumulation when insertion rates exceed deletion capacity. In some cases, TTL operations have been observed causing CPU spikes and query latency increases, particularly during peak hours . The GitHub discussion notes that TTL indexes should only be used with collections of moderate size due to these performance implications . For distributed lock implementations, MongoDB explicitly avoids using TTL indexes for lease management. Instead, it uses a separate `config.lockpings` collection with client-driven heartbeat updates . This design addresses critical TTL limitations: the 60-second granularity is too coarse for reliable lock expiration, and TTL cannot distinguish between a crashed client and a healthy client experiencing temporary GC pause or network delay. The lock can only be overtaken after 15 minutes without a heartbeat, providing much safer semantics . For large-scale TTL usage, several optimization approaches exist. Consider time-based partitioning (sharding by date) and dropping entire partitions instead of relying on document-level TTL . Stagger insertion times to avoid having all documents expire simultaneously, which helps smooth out TTL workload. For session data where 60-second granularity is acceptable, TTL works well; for more precise expiration requirements, implement application-level expiration logic. Always monitor TTL deletion rates using `db.serverStatus().metrics.ttl` to ensure the background thread keeps up with insertion rates .

How do TTL (Time-To-Live) indexes work and what are their limitations?

TTL indexes in MongoDB provide automatic document expiration and deletion based on a date field, using a background thread that runs every 60 seconds, but come with significant limitations including non-immediate deletion, compound index restrictions, and potential performance impacts.

How do TTL (Time-To-Live) indexes work and what are their limitations?

TTL indexes in MongoDB provide automatic document expiration and deletion based on a date field, using a background thread that runs every 60 seconds, but come with significant limitations including non-immediate deletion, compound index restrictions, and potential performance impacts.

Scenario Questions

Follow-up Questions