beginnerintermediateadvanced

Python · Performance

Profiling tools

Concepts: 3

timeit for comparing small snippets, cProfile/pstats for finding what actually dominates a whole program's runtime, and tracemalloc/memory profilers/py-spy/APM for measuring and observing memory and running processes.

This section

A Simple & Effective Way To Improve Python Class PerformanceArjanCodes

Measuring instead of guessing

The real tools that replace intuition about performance with actual measurements — from a single snippet to a whole running process.

timeit and microbenchmarks

coreintermediate

timeit.timeit(stmt, setup, number=N) runs a snippet N times and returns the total time, averaging out noise. It is the right tool for comparing two small alternatives — never for timing a whole application, where cProfile is better.

Think of it as

A single time.perf_counter() measurement is one photograph — it can be blurry from a random system hiccup. timeit is averaging many photographs together, which is why it is trusted for comparing two small snippets where the difference might be tiny.

python

import timeit

t = timeit.timeit(
    stmt="x in s",
    setup="s = set(range(1000)); x = 999",
    number=10000,
)

What we're doing: Compare two ways of building a list of squares and measure which one timeit actually reports as faster.

timeit_comparison.pypython

import timeit

t_loop = timeit.timeit("[i * i for i in range(1000)]", number=10000)
t_map = timeit.timeit("list(map(lambda i: i * i, range(1000)))", number=10000)

print(f"list comprehension: {t_loop:.4f}s")
print(f"map + lambda:       {t_map:.4f}s")

3: A list comprehension, run 10,000 times — timeit sums the total time across all runs.
4: The same job via map() and a lambda, timed the same way for a fair comparison.

Output

list comprehension: 0.5250s
map + lambda:       0.9488s

Why this works: Both produce the identical list of squares — timeit's repeated runs (10,000 here) average out any single-run noise, making the real, consistent difference between the two approaches visible rather than lost in measurement jitter.

Trusting a single time.perf_counter() measurement over timeit

Wrong

python

start = time.perf_counter()
result = [i * i for i in range(1000)]
elapsed = time.perf_counter() - start
print(elapsed)   # one sample -- could be a fluke, system noise, or a GC pause

Better

python

t = timeit.timeit("[i * i for i in range(1000)]", number=10000)
print(t / 10000)   # averaged over 10,000 runs -- noise washes out

What you see: Two runs of the same benchmark give noticeably different results, making it impossible to trust a conclusion drawn from either one.

Why: A single measurement can be skewed by a background process, a garbage collection pause, or CPU frequency scaling — timeit's repeated-run averaging (and disabling the GC by default) is specifically designed to cancel out exactly this kind of one-off noise.

Only stmt is timed, on every iteration

setup runs ONCE

s = set(range(1000)); x = 999 — before timing starts

stmt runs number= times

x in s — timed on every single iteration

Total time returned

summed across all runs — divide by number for per-run average

Noise averages out

a single perf_counter() call can be skewed by one GC pause; 10,000 runs is not

timeit — the interfaces worth knowing

timeit — the interfaces worth knowing
Form	Use
timeit.timeit(stmt, setup, number=N)	from Python code, returns total seconds for N runs
python -m timeit "expr"	from the command line, auto-picks a repeat count
%timeit expr	in a Jupyter/IPython cell, with automatic statistics

Together

python

import timeit

t = timeit.timeit("x in s", setup="s = set(range(1000)); x = 999", number=10000)
print(f"{t:.5f}s for 10000 runs")

Remember: timeit.timeit(stmt, setup, number=N) runs a snippet N times, averaging out noise — for small alternatives only.

cProfile and pstats

coreintermediate

cProfile.run("main()") runs a program under a profiler that records how many times every function was called and how long each took. pstats.Stats reads that data and prints it sorted however is useful — by total time, by calls, and so on.

Think of it as

timeit is a stopwatch for one small snippet you already suspect is slow. cProfile is a full itemized receipt for an entire program run — it tells you WHICH function actually consumed the time, without having to guess which part to time first.

python

python -m cProfile -s cumulative myapp.py

# or from code:
import cProfile, pstats
cProfile.run("main()", "output.prof")
pstats.Stats("output.prof").sort_stats("cumulative").print_stats(5)

What we're doing: Profile a real program with two functions of very different cost and read pstats' output to identify which one actually dominates.

profile_demo.pypython

import cProfile
import pstats

def slow_function():
    total = 0
    for i in range(1_000_000):
        total += i
    return total

def fast_function():
    return sum(range(1_000_000))

def main():
    slow_function()
    fast_function()

cProfile.run("main()", "profile_output.prof")
stats = pstats.Stats("profile_output.prof")
stats.sort_stats("cumulative")
stats.print_stats(5)

17: cProfile.run records every function call's timing while main() runs, saving it to profile_output.prof.
19: sort_stats("cumulative") orders the report by total time including nested calls — the most useful default for finding the real bottleneck.

Output

         7 function calls in 0.060 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.060    0.060 {built-in method builtins.exec}
        1    0.000    0.000    0.060    0.060 <string>:1(<module>)
        1    0.000    0.000    0.060    0.060 profile_demo.py:16(main)
        1    0.047    0.047    0.047    0.047 profile_demo.py:5(slow_function)
        1    0.000    0.000    0.013    0.013 profile_demo.py:12(fast_function)

Why this works: The pstats report shows slow_function at 0.047s tottime versus fast_function at 0.013s cumtime — even without knowing which function was "supposed" to be slow beforehand, the profiler's own numbers point directly at the manual-loop version as the real cost, exactly the sum()-versus-hand-loop difference many teams guess about instead of measuring.

Reading tottime as if it were the whole story

Wrong

python

# a function with LOW tottime but HIGH cumtime -- easy to miss
def orchestrator():
    for item in items:
        do_expensive_work(item)   # the REAL cost is hidden in here

# sorted by tottime alone, orchestrator() looks cheap

Better

python

stats.sort_stats("cumulative")   # includes time spent in CALLED functions
stats.print_stats(10)
# now orchestrator() correctly shows as expensive, via its cumtime

What you see: A profiler report sorted by tottime makes an "orchestrator" function that mostly calls other functions look cheap, hiding where the real cost actually lives.

Why: tottime only counts time spent directly inside a function's own code, excluding anything it calls — a thin wrapper around expensive work has near-zero tottime but high cumtime. Sorting by cumulative time is what surfaces the real cost of a call chain, not just individual functions in isolation.

From a running program to a ranked bottleneck list

cProfile.run("main()", "out.prof")

records ncalls, tottime, cumtime per function

pstats.Stats("out.prof")

loads the raw stats back for analysis

stats.sort_stats("cumulative")

orders by own time PLUS everything called — not tottime alone

stats.print_stats(N)

top N entries — the real bottleneck is at the top

pstats columns

pstats columns
Column	Means
ncalls	how many times the function was called
tottime	total time spent in the function itself, excluding calls to other functions
cumtime	total time including everything the function called
percall	tottime or cumtime divided by ncalls

Together

python

import cProfile
cProfile.run("main()", "output.prof")

import pstats
stats = pstats.Stats("output.prof")
stats.sort_stats("cumulative").print_stats(10)

Remember: cProfile.run() profiles a whole program; pstats.Stats reads the results — sort by cumulative time, not tottime alone.

tracemalloc, memory profilers, py-spy, and APM

coreintermediate

tracemalloc tracks memory allocations and reports current/peak usage. memory_profiler adds a line-by-line report. py-spy profiles a running process from OUTSIDE it, no code changes — APM tools do the same continuously in production.

Think of it as

tracemalloc is stepping on a scale before and after eating — a before/after measurement you have to trigger yourself, inside the code. py-spy is watching someone eat through a window — it profiles a process from the outside, without needing to modify anything running inside it.

python

import tracemalloc

tracemalloc.start()
# ... code to measure ...
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()

What we're doing: Measure the real memory cost of building a one-million-item list with tracemalloc.

tracemalloc_demo.pypython

import tracemalloc

tracemalloc.start()
data = [i for i in range(1_000_000)]
current, peak = tracemalloc.get_traced_memory()
print(f"current: {current / 1024 / 1024:.2f} MB, peak: {peak / 1024 / 1024:.2f} MB")
tracemalloc.stop()

3: tracemalloc.start() begins tracking every allocation from this point forward.
5: get_traced_memory() returns real, measured byte counts — not an estimate based on object count.

Output

current: 38.56 MB, peak: 38.56 MB

Why this works: A million-item Python list of small integers genuinely costs tens of megabytes — each element is a full Python object with overhead, not a raw 8-byte machine integer. tracemalloc measures this directly rather than requiring a guess based on "how many items are in it."

Leaving tracemalloc running in production by default

Wrong

python

# at the top of the app, unconditionally:
import tracemalloc
tracemalloc.start()   # now running for the ENTIRE process lifetime

Better

python

# only when actually debugging a memory issue:
if os.environ.get("DEBUG_MEMORY"):
    tracemalloc.start()
    atexit.register(lambda: print(tracemalloc.get_traced_memory()))

What you see: The application runs measurably slower and uses more memory than before, for no visible benefit most of the time.

Why: tracemalloc adds real per-allocation overhead to track every object — it is a debugging tool meant to be turned on for a specific investigation, not left running unconditionally as a permanent production default.

How each tool attaches to a running program

tracemalloc

code you add/remove yourself

memory_profiler

@profile decorator, line-by-line

py-spy

attaches externally — no code changes

APM (Datadog, etc.)

external agent, continuous, in production

Memory/process profiling tools by how they attach

Memory/process profiling tools by how they attach
Tool	Attaches by
tracemalloc	code you add and remove yourself (tracemalloc.start()/stop())
memory_profiler	a @profile decorator on the function to inspect
py-spy	external — attaches to an already-running process, no code changes
APM (Datadog, etc.)	external agent, running continuously in production

Together

python

import tracemalloc

tracemalloc.start()
data = [i for i in range(1_000_000)]
current, peak = tracemalloc.get_traced_memory()
print(f"current: {current / 1024 / 1024:.2f} MB, peak: {peak / 1024 / 1024:.2f} MB")
tracemalloc.stop()

Remember: tracemalloc measures memory from inside code you control; py-spy/APM profile a process from outside, no code changes.