Concurrency · What a 5-Year Python Engineer Should Be Able to Explain · Python

Choosing a concurrency model under real constraints

standardadvanced

Given a real workload, the choice reduces to two questions asked in order: is the work CPU-bound or I/O-bound, and does it need to talk to a large number of things at once? CPU-bound work needs multiprocessing (the GIL blocks real parallelism from threads); I/O-bound work with a handful of connections is fine with threading; I/O-bound work with thousands of concurrent connections is where asyncio's single-threaded event loop actually pays for itself.

Think of it as

Threading, multiprocessing, and asyncio are three different tools solving three different bottlenecks, not three interchangeable ways to do the same thing faster. Multiprocessing buys real parallel CPU cores by paying for separate processes. Threading buys concurrency during I/O waits without that process overhead, but the GIL still serializes actual Python bytecode. Asyncio buys concurrency at a MUCH larger scale (thousands of connections) by never blocking the single thread at all — at the cost that every blocking call anywhere in that thread stalls every other coroutine waiting on the event loop.

python

def choose_model(cpu_bound, concurrent_operations):
    if cpu_bound:
        return "multiprocessing"
    return "asyncio" if concurrent_operations > 100 else "threading"

What we're doing: Apply a two-question decision rule (CPU-bound? how many concurrent operations?) to four real workload descriptions, and confirm it recommends a different model for each distinct case.

choose_concurrency_model.pypython

def choose_model(description, cpu_bound, concurrent_operations):
    if cpu_bound:
        model = "multiprocessing"
    elif concurrent_operations > 100:
        model = "asyncio"
    else:
        model = "threading"
    return f"{description}: {model}"


workloads = [
    ("Resize 50 images (CPU-heavy)", True, 50),
    ("Fetch 10 internal microservice URLs", False, 10),
    ("Proxy 5,000 concurrent WebSocket connections", False, 5000),
    ("Parse and hash 1M records for a nightly batch job", True, 1),
]
for description, cpu_bound, concurrent_operations in workloads:
    print(choose_model(description, cpu_bound, concurrent_operations))

2: CPU-bound is checked FIRST — no I/O-concurrency number can compensate for the GIL blocking real parallelism on CPU-heavy work.
4: Only once CPU-bound is ruled out does the concurrency SCALE decide threading vs asyncio — a small number of concurrent I/O operations does not need an event loop.

Output

Resize 50 images (CPU-heavy): multiprocessing
Fetch 10 internal microservice URLs: threading
Proxy 5,000 concurrent WebSocket connections: asyncio
Parse and hash 1M records for a nightly batch job: multiprocessing

Why this works: The last workload has concurrent_operations=1 (a single sequential batch job) but is still routed to multiprocessing — because cpu_bound is checked first, and no amount of low I/O-concurrency changes the fact that hashing 1M records needs real parallel CPU cores, not I/O-wait-driven concurrency. The two questions are asked in a fixed order for exactly this reason.

Reaching for asyncio because it "sounds modern," on a CPU-bound workload

Wrong

python

import asyncio

async def hash_record(record):
    return hash(record)   # pure CPU work -- awaiting it buys NOTHING

async def hash_all(records):
    return await asyncio.gather(*(hash_record(r) for r in records))
    # still runs on ONE thread, one core -- no faster than a plain loop

Better

python

from concurrent.futures import ProcessPoolExecutor

def hash_all(records):
    with ProcessPoolExecutor() as pool:
        return list(pool.map(hash, records))   # actually uses multiple cores

What you see: The "async" version runs at roughly the same speed as a plain synchronous loop over the same records — sometimes slightly slower, once the event-loop scheduling overhead is counted — because nothing in the workload ever awaits real I/O.

Why: asyncio's entire benefit comes from NOT blocking the thread while waiting on I/O — a pure CPU computation like hash() never actually waits on anything, so wrapping it in async/await adds coroutine and event-loop overhead without buying any concurrency. Only ProcessPoolExecutor (or multiprocessing directly) escapes the GIL to use more than one core for CPU-bound work.

Remember: Ask CPU-bound-or-I/O-bound first (multiprocessing wins CPU-bound, no exceptions); ask how many concurrent operations second (asyncio at real scale, threading otherwise) — and remember one blocking call anywhere in an async app stalls every coroutine sharing that event loop.