beginnerintermediateadvanced

Python · Profiling tools

Profiling tools

timeit for small snippets, cProfile/pstats for whole programs, tracemalloc for memory — all verified with real measurements

timeitaverages many runs to cancel out single-measurement noise
cProfile + pstatsfinds which function actually dominates runtime
tracemallocmeasures real memory allocations, current and peak

1.timeit and microbenchmarks

timeit.timeit(stmt, setup, number=N) runs a snippet N times and returns the total time, averaging out noise. It is the right tool for comparing two small alternatives — never for timing a whole application, where cProfile is better.

Key features

timeit.timeit(stmt, setup, number=1000) runs the statement 1000 times, returns total time
The setup argument runs once, before timing starts — builds test data without counting it
timeit disables the garbage collector during timing, for more consistent results
python -m timeit "x in s" runs from the command line, without writing a script
timeit is for comparing SMALL snippets — a whole real application needs cProfile instead

timeit — the interfaces worth knowing

timeit — the interfaces worth knowing
Form	Use
timeit.timeit(stmt, setup, number=N)	from Python code, returns total seconds for N runs
python -m timeit "expr"	from the command line, auto-picks a repeat count
%timeit expr	in a Jupyter/IPython cell, with automatic statistics

Only stmt is timed, on every iteration

setup runs ONCE

s = set(range(1000)); x = 999 — before timing starts

stmt runs number= times

x in s — timed on every single iteration

Total time returned

summed across all runs — divide by number for per-run average

Noise averages out

a single perf_counter() call can be skewed by one GC pause; 10,000 runs is not

Command

python

import timeit

t = timeit.timeit(
    stmt="x in s",
    setup="s = set(range(1000)); x = 999",
    number=10000,
)

timeit.timeit(stmt, setup, number=N) runs a snippet N times, averaging out noise — for small alternatives only.

2.cProfile and pstats

cProfile.run("main()") runs a program under a profiler that records how many times every function was called and how long each took. pstats.Stats reads that data and prints it sorted however is useful — by total time, by calls, and so on.

Key features

cProfile.run("main()", "output.prof") profiles main() and saves the raw stats to a file
pstats.Stats("output.prof") loads those stats back for analysis
stats.sort_stats("cumulative") sorts by own time plus everything it called
stats.print_stats(N) prints the top N entries by whatever sort was chosen
tottime is time spent IN a function itself; cumtime includes functions it called

pstats columns

pstats columns
Column	Means
ncalls	how many times the function was called
tottime	total time spent in the function itself, excluding calls to other functions
cumtime	total time including everything the function called
percall	tottime or cumtime divided by ncalls

From a running program to a ranked bottleneck list

cProfile.run("main()", "out.prof")

records ncalls, tottime, cumtime per function

pstats.Stats("out.prof")

loads the raw stats back for analysis

stats.sort_stats("cumulative")

orders by own time PLUS everything called — not tottime alone

stats.print_stats(N)

top N entries — the real bottleneck is at the top

Command

python

python -m cProfile -s cumulative myapp.py

# or from code:
import cProfile, pstats
cProfile.run("main()", "output.prof")
pstats.Stats("output.prof").sort_stats("cumulative").print_stats(5)

cProfile.run() profiles a whole program; pstats.Stats reads the results — sort by cumulative time, not tottime alone.

3.tracemalloc, memory profilers, py-spy, and APM

tracemalloc tracks memory allocations and reports current/peak usage. memory_profiler adds a line-by-line report. py-spy profiles a running process from OUTSIDE it, no code changes — APM tools do the same continuously in production.

Key features

tracemalloc.start() begins tracking; get_traced_memory() returns (current, peak) in bytes
@profile from memory_profiler reports memory usage line by line for a decorated function
py-spy dump --pid <pid> profiles a running process from outside it, no restart needed
APM tools (Datadog, New Relic) continuously profile production, not just one session
tracemalloc has real overhead — enable it for debugging, not as an always-on default

Memory/process profiling tools by how they attach

Memory/process profiling tools by how they attach
Tool	Attaches by
tracemalloc	code you add and remove yourself (tracemalloc.start()/stop())
memory_profiler	a @profile decorator on the function to inspect
py-spy	external — attaches to an already-running process, no code changes
APM (Datadog, etc.)	external agent, running continuously in production

How each tool attaches to a running program

tracemalloc

code you add/remove yourself

memory_profiler

@profile decorator, line-by-line

py-spy

attaches externally — no code changes

APM (Datadog, etc.)

external agent, continuous, in production

Command

python

import tracemalloc

tracemalloc.start()
# ... code to measure ...
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()

tracemalloc measures memory from inside code you control; py-spy/APM profile a process from outside, no code changes.

Spotted a mistake on this page? Tell us.

Read Sheet