beginnerintermediateadvanced

Python · Optimization

Optimization

Caching, batching, parallelization, and the rule that ties every technique in this section together

@lru_cachemeasured 370x speedup on recursive Fibonacci
Batchingfewer, larger round trips beat many small ones
Parallelization vs. async I/OCPU-bound cores vs. I/O-bound overlap
Measure firstintuition about "what is slow" is frequently wrong

1.Caching and lazy evaluation

@functools.lru_cache remembers a function's past results, so calling it again with the same arguments returns instantly. Lazy evaluation delays work until actually needed — sometimes it never is, avoiding it entirely.

Key features

@lru_cache(maxsize=None) caches every distinct set of arguments a function was called with
A cached function must be pure — caching one with side effects causes bugs
lru_cache has real memory cost — every cached result stays until evicted or cleared
A generator expression computes items on demand — a list comprehension computes it all now
Lazy evaluation avoids unnecessary work; caching avoids repeated work — different savings

lru_cache — the interface worth knowing

lru_cache — the interface worth knowing
Call	Effect
@lru_cache(maxsize=None)	unbounded cache — every distinct call is remembered forever
@lru_cache(maxsize=128)	bounded cache — oldest unused entries evicted once full
fn.cache_info()	reports hits, misses, and current cache size
fn.cache_clear()	empties the cache manually

Caching turns exponential re-work into linear work

yesno

fib_cached(28)

n already in cache?

return cached result

instant — 0.000175s total

compute fib(n-1) + fib(n-2)

once per distinct n only

store in cache, return

Command

python

from functools import lru_cache

@lru_cache(maxsize=None)
def expensive_computation(x):
    ...

@lru_cache remembers past results for pure functions but never detects staleness; lazy evaluation avoids unneeded work.

2.Batching, connection pooling, and query optimization

Batching combines many small operations into fewer, larger ones. Connection pooling reuses already-open connections instead of opening a new one per request. Query optimization means a query (and its indexes) doing less work to answer.

Key features

A round trip has fixed overhead PER call — fewer, larger calls amortize that cost
The N+1 query problem: fetching N rows with N queries instead of one batch — an ORM trap
A connection pool keeps open connections ready, avoiding handshake cost per request
A missing index turns a query into a full table scan — often the largest single win
Batching trades per-item latency for throughput — a big batch finishes far sooner overall

Batched vs. unbatched, by round trips

Batched vs. unbatched, by round trips
Approach	Round trips for 100 items
One query per item (N+1)	100 separate round trips
One batched query	1 round trip, WHERE id IN (...)
No connection pool	connection handshake cost paid on every request
Connection pool	handshake cost paid once, connections reused

N round trips vs. one batched round trip

N+1 (unbatched)

One query PER item, inside a loop
100 items = 100 round trips
Fixed per-call overhead paid every time

Batched (WHERE id IN (...))

One query for every item at once
100 items = 1 round trip
Fixed overhead paid exactly once

Command

python

# batched query, one round trip instead of N
users = db.query("SELECT * FROM users WHERE id IN (?)", user_ids)

# connection pool, reused across requests
pool = create_pool(min_size=5, max_size=20)
async with pool.acquire() as conn:
    await conn.fetch(...)

Batching combines many round trips into fewer, larger ones; watch for the N+1 pattern hiding inside a loop.

3.Parallelization, async I/O, and serialization optimization

Parallelization runs independent work across CPU cores at once. Async I/O overlaps many waits on one thread instead of one at a time. Serialization optimization means a faster format or less data — json can be a real cost at scale.

Key features

Parallelization (multiprocessing) uses multiple CPU cores — right for CPU-bound work
Async I/O overlaps waits on one thread — right for many concurrent I/O-bound waits
json.dumps/loads has a real cost at scale — not free just because the syntax is simple
orjson and similar C-accelerated libraries serialize JSON significantly faster than json
Serializing LESS data (only needed fields) often beats switching serializers

Choosing between parallelization and async I/O

Choosing between parallelization and async I/O
Workload	Fits
CPU-bound (heavy computation)	multiprocessing (ProcessPoolExecutor)
I/O-bound, many concurrent waits	asyncio
I/O-bound, few concurrent waits, sync libraries	threading

ProcessPoolExecutor vs. asyncio.gather, on I/O-bound work

ProcessPoolExecutor (wrong fit here)

Real spawn + pickling overhead per process
Measured 0.454s — SLOWER than serial (0.405s)
Right for CPU-bound work, not for waiting

asyncio.gather (right fit)

Overlaps the same waits on one thread
No process-spawn overhead at all
Right for many concurrent I/O-bound waits

Command

python

from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor() as ex:
    results = list(ex.map(cpu_heavy_function, items))

import json
data = json.dumps(obj)   # standard library, simple, not the fastest option

Parallelization uses multiple cores for CPU-bound work; async I/O overlaps waits for I/O-bound work; serialization is not free.

4.Measure first, optimize second

Intuition about what is slow in a program is frequently wrong — profile with a real tool (cProfile, timeit) before spending effort optimizing, or the effort often lands on something that was never the actual bottleneck.

Key features

Human intuition about "what is slow" is frequently wrong — profile instead of guessing
cProfile shows which function dominates runtime — often not the "complex-looking" one
Optimizing a function that is 1% of runtime cannot meaningfully improve performance
A profiler's numbers are specific to the workload — profile with realistic data, not a toy
The rule applies at every scale — a microbenchmark or a whole app needs measurement first

Guessed vs. measured — a real contrast

Guessed vs. measured — a real contrast
Belief	What profiling often reveals
"The nested loop must be the bottleneck"	Sometimes it is 1% of runtime; the real cost is a database call
"This function looks complex, it must be slow"	Complexity of CODE and complexity of RUNTIME COST are different things
"I optimized the algorithm, it must be faster now"	Sometimes the real cost was I/O, untouched by the optimization

Looks complex vs. actually costs time

Intuition: "the loop is slow"

cProfile.run("main()")

looks_complex_but_is_cheap()

cumtime 0.001s

looks_simple_but_is_expensive()

cumtime 0.100s — the real cost

Command

python

# 1. Profile first
python -m cProfile -s cumulative myapp.py

# 2. Find what ACTUALLY dominates runtime (top of the sorted list)
# 3. Optimize THAT, specifically
# 4. Re-profile to confirm the fix actually helped

Intuition about what is slow is frequently wrong — profile first, optimize the actual bottleneck, then re-profile.

Spotted a mistake on this page? Tell us.

Read Sheet