What tools would you use to profile a slow Python function, and how do you interpret the output?
Start with timeit for micro-benchmarks, cProfile for call-level hotspots, line_profiler for per-line detail
Pick the tool based on scope. For a small piece of code, timeit gives repeatable timings with minimal setup. For a whole program or function, cProfile tells you where time is spent across calls; sort by cumulative time to find the expensive subtree and by tottime to find the function doing the real work. py-spy attaches to a running process without restarting it, which is invaluable in production. For line-level detail inside one hot function, line_profiler tells you exactly which line dominates. The interpretation rules matter: ncalls and tottime point to the culprit, cumtime identifies the subtree, and a high number of primitive calls often means an algorithmic problem rather than a constant-factor problem.
- 1
timeit: micro-benchmarks; use the command line form for a fair comparison across variants.
- 2
cProfile: call-level profiling; run python -m cProfile -s cumtime script.py, or profile programmatically.
- 3
py-spy: sampling profiler that attaches to a live process and works in production without code changes.
- 4
line_profiler: per-line detail for one function once you already know where to look.
- 5
memory_profiler and tracemalloc for memory, as a separate axis from time.
- 6
Common mistake: profiling with a debugger attached or with print statements in the loop, which distorts results.
- 7
Common mistake: optimizing the top function by tottime when the real cost is in a call it makes. Check cumtime before you refactor.
- 8
Version note: cProfile has been in the standard library since Python 2.5. py-spy is third-party and works across 3.x versions.
Scenario Questions
0-2 years experience
- 1You want to compare two implementations of the same function. Which standard library tool?
- 2What does cProfile produce and how do you sort the output?
2-5 years experience
- 1cProfile shows a function with high tottime but low ncalls. What does that suggest?
- 2You cannot restart production to profile. Which tool attaches to the running process?
5-8 years experience
- 1Profiling overhead distorts your measurement for a tight loop. How do you get a fair comparison?
- 2Most of the runtime is in a C extension. How do you profile it effectively?
8+ years experience
- 1Design a continuous profiling strategy for a fleet of services that captures regressions before they reach users.
- 2Explain how to combine CPU profiling, memory profiling, and trace data to attribute cost across a distributed request path.
Follow-up Questions
- Why should you not profile with a debugger attached?
- How would you profile an async service where most time is spent awaiting IO?