Your program is consuming increasing memory over time (a suspected memory leak) despite Python's garbage collector. How would you investigate this?
Snapshot with tracemalloc, inspect gc.get_objects(), find the growth, then break the retention
Python's GC handles cycles, so a growing RSS means something is keeping objects alive that you did not intend. The classic causes are module-level caches with no bound, event handlers or callbacks registered and never removed, closures capturing large objects, thread locals, and references held by C extensions. The systematic approach is to measure first: use tracemalloc to take snapshots at intervals and compare the top allocation sites, which shows which lines are responsible for growth. Then use gc.get_objects() and gc.get_referrers() to find who is holding a growing object. For long-lived services, a heap snapshot diff is the fastest path. Once you find the retention path, the fix is usually bounding the cache with an LRU, using weak references, or unregistering callbacks on teardown.
- 1
tracemalloc.start(), take two snapshots minutes apart, then snapshot.compare_to(previous, 'lineno') gives the top growth lines.
- 2
gc.get_objects() lets you count how many instances of a type exist; a steadily increasing count is a strong signal.
- 3
gc.get_referrers(obj) shows who holds a reference, which usually reveals the cache or registry keeping it alive.
- 4
objgraph and pympler give more readable object graphs and diff views.
- 5
weakref.WeakValueDictionary or WeakSet is the standard fix for caches that must not keep objects alive.
- 6
Common mistake: assuming gc.collect() will fix a leak. It only breaks cycles; it does not release objects still referenced.
- 7
Version note: tracemalloc is available since 3.4. gc.freeze() in 3.7 helps fork-based servers by moving startup objects out of GC scans.
Scenario Questions
0-2 years experience
- 1Which module in the standard library lets you track allocations by line number?
- 2Why does a module-level list that keeps growing cause a memory leak?
2-5 years experience
- 1A web service grows steadily until it is OOM-killed and restarted. How do you find which object count is growing?
- 2You cache results in a dict without a size limit and memory grows. What is the smallest fix?
5-8 years experience
- 1You suspect a C extension is leaking. What tools do you use and how do you confirm the leak is outside pure Python?
- 2Your service registers callbacks on each request and never removes them. How do you detect and fix this pattern at scale?
8+ years experience
- 1Design a long-running data pipeline that must run for weeks with bounded memory, and explain how you would prove the bound holds under production load.
- 2Explain how gc generations, gc.freeze(), and fork-based workers interact with memory growth, and how to tune them safely.
Follow-up Questions
- Why does gc.collect() not fix a typical leak?
- How would you diagnose a leak in a long-running async service?