How would you optimize a Python application that spends most of its time waiting on a database or network call?
Overlap the waiting with concurrency and reduce the number of round trips
If the application is waiting rather than computing, the goal is to overlap waits and reduce the number of waits. The first lever is to reduce round trips: batch queries, use joins or IN clauses instead of N+1 loops, cache hot reads, and select only the columns you need. The second lever is concurrency: an async stack with an async database driver and HTTP client lets one thread keep hundreds of requests in flight, and a thread pool with a synchronous driver is the pragmatic alternative when async drivers are not available. The third lever is pooling: reuse connections and configure pool sizes and timeouts to match the backend's limits. The fourth is timeouts and retries with backoff so slow calls fail fast instead of consuming worker capacity.
- 1
Measure first. Identify whether time is in the database, the network, or serialization, using query logs, tracing, and profiling.
- 2
Batch and cache: the fastest call is the one you do not make.
- 3
Use async drivers or a thread pool. Do not call a blocking driver from an async handler.
- 4
Configure connection pools explicitly: max size, idle timeout, and statement timeouts.
- 5
Add timeouts on every external call. A missing timeout is a production outage waiting to happen.
- 6
Trade-off: higher concurrency improves throughput but increases load on downstream systems and complicates debugging.
- 7
Common mistake: increasing worker count to mask a slow query. That multiplies load on the database.
- 8
Version note: asyncpg and psycopg 3 async support are the modern choices for PostgreSQL. httpx and aiohttp are standard async HTTP clients.
Scenario Questions
0-2 years experience
- 1Your endpoint calls the database ten times in a loop. What is the simplest improvement?
- 2Why does adding more threads to a database-heavy endpoint sometimes make it slower?
2-5 years experience
- 1You switch to an async driver and latency drops, but CPU rises. Why?
- 2You have a synchronous ORM in an async service. How do you integrate it without blocking the loop?
5-8 years experience
- 1Your service is fast at low load and collapses at high load. How do you find the bottleneck and apply back-pressure?
- 2You need to fan out to five downstream services with per-call timeouts and partial-failure handling. How do you structure it?
8+ years experience
- 1Design an IO-heavy service with end-to-end latency and throughput targets, including caching, pooling, timeouts, retries, and observability.
- 2Explain how to choose between async IO, thread pools, and a message queue for a mixed workload, and how to migrate incrementally.
Follow-up Questions
- How do you size a connection pool correctly for a given backend limit?
- What is the N+1 query problem and how do you detect it in production?