How would you run CPU-bound work alongside an asyncio application without blocking the event loop?
Offload CPU work to a process pool via run_in_executor or asyncio.to_thread
The event loop is single-threaded, so any CPU-bound code inside an async function stalls every other task. The fix is to hand that work to a worker that does not run on the loop thread. For CPU-bound work, use a ProcessPoolExecutor and await loop.run_in_executor(pool, func, *args) or wrap it in an async helper. A ThreadPoolExecutor helps only for blocking IO or for C code that releases the GIL; for pure Python CPU work it will just contend on the GIL. The important design points are: bound the pool size to the available cores, avoid passing huge objects across process boundaries because they are pickled, and apply back-pressure so the queue of submitted work does not grow without bound. In 3.9+ asyncio.to_thread is a convenient shortcut for the blocking-IO case.
- 1
ProcessPoolExecutor for CPU-bound work; ThreadPoolExecutor only for blocking IO or GIL-releasing C calls.
- 2
loop.run_in_executor is awaitable and integrates cleanly with gather, timeouts, and cancellation.
- 3
Bound the pool size to os.cpu_count() or a configured limit; the default can overwhelm the machine.
- 4
Pickle cost is real. Chunk data and pass file paths or shared memory, not multi-megabyte objects.
- 5
Apply back-pressure with an asyncio.Semaphore or a bounded queue so you do not submit faster than the pool can drain.
- 6
Common mistake: passing an open file object, a socket, or a database connection into a process. They are not picklable or meaningful in the child.
- 7
Common mistake: creating a new pool per request. Pool startup is expensive; create one pool for the service lifetime.
- 8
Version note: asyncio.to_thread was added in 3.9 as a convenience wrapper. ProcessPoolExecutor has been in concurrent.futures since 3.2.
Scenario Questions
0-2 years experience
- 1An async handler calls a CPU-heavy function and all other requests slow down. Why?
- 2Which executor type do you use for CPU-bound work?
2-5 years experience
- 1You offload work to a process pool but pass a 500 MB list and see latency explode. What is happening?
- 2You create a new ProcessPoolExecutor for each request and see high CPU on idle. How do you fix it?
5-8 years experience
- 1You need to bound the number of outstanding CPU jobs to protect the machine. How do you enforce back-pressure?
- 2A CPU job raises an exception inside the pool. How do you propagate it correctly to the awaiting coroutine and log it once?
8+ years experience
- 1Design an async service that mixes IO, CPU, and GPU work with bounded queues, cancellation, timeouts, and per-stage observability.
- 2Explain how to use shared memory or zero-copy approaches to avoid pickling overhead between asyncio and worker processes.
Follow-up Questions
- Why is a thread pool usually the wrong choice for CPU-bound Python work?
- How do you cancel an in-flight process pool task?