Filter concepts by levelShowing all levels.

Python · Standard Library

Standard Library

The modules a strong Python engineer reaches for instead of reinventing them

  • collectionsCounter, defaultdict, namedtuple, deque
  • itertools + functoolslazy iteration and function tools, no third-party libs needed
  • pathlib.Pathobject-oriented filesystem paths, cross-platform
  • asyncio + concurrent.futuresI/O-bound concurrency and thread/process pools

1.collections

collections supplies specialized containers — Counter, defaultdict, namedtuple, deque — that replace common hand-written patterns built from plain dicts and lists.

Key features

  • Counter(iterable) tallies items; .most_common(n) returns the top n as (item, count) pairs
  • defaultdict(factory) calls factory() to create a value the first time a key is missing
  • namedtuple builds a tuple subclass with named, dot-accessible fields — still immutable
  • deque supports O(1) append/pop from both ends; a list is O(n) at the front
  • ChainMap searches several dicts in order without copying or merging them

collections — the containers worth knowing

collections — the containers worth knowing
NameWhat it is for
Countera dict subclass that counts hashable items; most_common(n) ranks them
defaultdict(factory)a dict that runs factory() to create a missing key instead of raising KeyError
namedtuple(name, fields)a tuple subclass with named fields — Point(x, y).x instead of p[0]
dequea double-ended queue — appendleft/popleft are O(1), unlike a list
OrderedDicta dict that remembers insertion order and adds move_to_end()
ChainMap(*maps)views several dicts as one, checked in order, without copying
Reach for the specialized container that matches the problem

Counting items

Counter

Grouping into a dict

defaultdict(list)

Named fields

namedtuple

Fast both ends

deque

  1. Counting items — Counter
  2. Grouping into a dict — defaultdict(list)
  3. Named fields — namedtuple
  4. Fast both ends — deque

Command

python
from collections import Counter, defaultdict, namedtuple, deque

Counter(iterable)
defaultdict(factory)
namedtuple("Name", ["field1", "field2"])
deque(iterable, maxlen=None)

Counter tallies, defaultdict auto-creates missing values, namedtuple names fields, deque is fast at both ends — pick the one matching the shape of the problem.

2.itertools

itertools provides fast, memory-efficient building blocks for looping — chaining several iterables, slicing lazily, grouping consecutive items, and generating combinations — without building an intermediate list.

Key features

  • chain(*iterables) walks several iterables as if they were one, lazily
  • islice(iterable, stop) or (start, stop, step) slices any iterator, not just sequences
  • groupby(iterable, key) groups only CONSECUTIVE matching items — sort first if needed
  • product/permutations/combinations generate combinatorics without nested loops
  • count(start, step) is an infinite counter — always pair it with islice or a break

itertools — the functions worth knowing

itertools — the functions worth knowing
FunctionWhat it does
chain(*iterables)walks each iterable in turn as a single stream
islice(it, stop)lazily slices an iterator — works on generators, not just lists
groupby(it, key)groups consecutive items sharing the same key(item)
product(*iterables)the Cartesian product — every combination, order matters
combinations(it, r)every r-length selection, order does not matter, no repeats
count(start, step)an infinite arithmetic sequence — pair with islice or break
chain, then groupby — only CONSECUTIVE keys merge

chain(log_a, log_b)

dev1, dev1, dev2, dev1 — one lazy stream

groupby(combined, key)

starts a new group every time the key changes

3 groups, not 2

the second dev1 splits off — dev2 interrupted the run

  1. chain(log_a, log_b) — dev1, dev1, dev2, dev1 — one lazy stream
  2. groupby(combined, key) — starts a new group every time the key changes
  3. 3 groups, not 2 — the second dev1 splits off — dev2 interrupted the run

Command

python
from itertools import chain, islice, groupby, product, combinations

chain(iter1, iter2, ...)
islice(iterable, stop)
groupby(iterable, key=None)

itertools chains and slices iterators lazily; groupby only groups items that are already adjacent — sort first if the matches are scattered.

3.functools

functools provides tools for working with functions themselves — caching a function's results, pre-filling some of its arguments, and preserving its name and docstring through a decorator.

Key features

  • @lru_cache(maxsize=N) remembers past return values by argument — same args, no recompute
  • reduce(fn, iterable, start) folds a sequence into one value, applying fn cumulatively
  • partial(fn, *args) pre-fills some arguments, returning a new callable for the rest
  • @wraps(fn) inside a decorator copies __name__/__doc__ from the wrapped function
  • @singledispatch picks an implementation based on the type of the first argument

functools — the functions worth knowing

functools — the functions worth knowing
NameWhat it does
@lru_cache(maxsize=128)caches a function's return value per distinct argument set
@cacheunbounded shortcut for @lru_cache(maxsize=None), Python 3.9+
reduce(fn, iterable, start)cumulatively applies fn(acc, item), folding to one value
partial(fn, *args, **kwargs)returns a new callable with some arguments pre-filled
@wraps(fn)copies __name__/__doc__ onto a wrapper — essential inside any decorator
@singledispatchdispatches to a type-specific implementation, registered separately
functools — five tools, five distinct jobs

@lru_cache

remembers past return values by argument

reduce(fn, it, start)

folds a sequence into one value

partial(fn, *args)

pre-fills arguments, returns a new callable

@wraps(fn)

preserves __name__/__doc__ through a decorator

@singledispatch

dispatches by the type of the first argument

  • @lru_cache — remembers past return values by argument
  • reduce(fn, it, start) — folds a sequence into one value
  • partial(fn, *args) — pre-fills arguments, returns a new callable
  • @wraps(fn) — preserves __name__/__doc__ through a decorator
  • @singledispatch — dispatches by the type of the first argument

Command

python
from functools import lru_cache, reduce, partial, wraps

@lru_cache(maxsize=128)
def expensive(n): ...

reduce(function, iterable, initial)
partial(function, *fixed_args)

@lru_cache trades memory for speed on pure functions with hashable arguments; @wraps belongs on every decorator's inner function, without exception.

4.pathlib

pathlib represents a filesystem path as a Path object instead of a string — join pieces with /, read properties like .suffix and .parent, and call methods like .exists() directly on it.

Key features

  • Path("a") / "b" / "c.txt" joins path segments — works the same on every OS
  • .name, .stem, .suffix, .parent read pieces of a path without string slicing
  • .exists(), .is_file(), .is_dir() check the filesystem directly
  • .read_text()/.write_text() read or write a whole file in one call
  • .glob("*.py") lists matching files; Path.cwd() and Path.home() are common starting points

pathlib.Path — the surface worth knowing

pathlib.Path — the surface worth knowing
MemberWhat it does
Path(*parts) / "next"builds a path; / joins another segment onto it
.name / .stem / .suffixthe final component / that component without its extension / just the extension
.parent / .partsthe containing directory / a tuple of every segment
.exists() / .is_file() / .is_dir()checks against the real filesystem
.read_text() / .write_text(s)reads or writes an entire text file in one call
.glob(pattern) / .iterdir()matching files by pattern / every entry in a directory
.mkdir(parents=True, exist_ok=True)creates a directory, including missing parents
A Path's pieces

Path("data") / "reports" / "2026.csv"

Path("data")

root segment — the first component the path is built from

/ "reports"

join operator — / appends another path segment, OS-independent

"2026.csv"

final component — becomes .name; .stem is "2026", .suffix is ".csv"

  • Whole: Path("data") / "reports" / "2026.csv"
  • Path("data") — root segment: the first component the path is built from
  • / "reports" — join operator: / appends another path segment, OS-independent
  • "2026.csv" — final component: becomes .name; .stem is "2026", .suffix is ".csv"

Command

python
from pathlib import Path

p = Path("dir") / "file.txt"
p.exists()
p.read_text()

Path("a") / "b" joins segments correctly on every OS; use .suffix/.stem/.parent instead of slicing the string yourself.

5.datetime and zoneinfo

datetime represents dates, times, and durations as objects instead of strings; zoneinfo attaches a real named time zone (like "America/New_York") that correctly accounts for daylight saving time.

Key features

  • date(y, m, d) is calendar-only; datetime(y, m, d, h, m, s) adds a time of day
  • timedelta represents a duration; add/subtract it directly from a date or datetime
  • .strftime(fmt) formats a datetime as a string; .strptime(s, fmt) parses one back
  • A naive datetime has no time zone attached; pass tzinfo= to make it aware
  • ZoneInfo("Region/City") from zoneinfo gives a real IANA zone, including DST rules

datetime and zoneinfo — the surface worth knowing

datetime and zoneinfo — the surface worth knowing
MemberWhat it does
date(year, month, day)a calendar date with no time component
datetime(y, m, d, h, mi, s, tzinfo=)a specific point in time, optionally time-zone aware
timedelta(days=, hours=, ...)a duration — add/subtract it from a date or datetime
.strftime(fmt) / .strptime(s, fmt)format a datetime to string / parse a string to datetime
.isoformat() / .fromisoformat(s)the standard "2026-08-21T14:30:00+00:00" form, both directions
ZoneInfo("America/New_York")a real IANA time zone, correctly applying DST rules for that date
timezone.utcthe fixed UTC offset — always available, no zone database needed
Fixed offset vs. ZoneInfo, same city

timezone(timedelta(hours=-4))

  • Never changes — baked in at creation time
  • Correct in August (EDT), wrong in January
  • Off by an hour once DST flips

ZoneInfo("America/New_York")

  • Consults the IANA database for the given date
  • Applies EDT in summer, EST in winter, automatically
  • Correct for any recurring or future-dated event
  • timezone(timedelta(hours=-4))
    • Never changes — baked in at creation time
    • Correct in August (EDT), wrong in January
    • Off by an hour once DST flips
  • ZoneInfo("America/New_York")
    • Consults the IANA database for the given date
    • Applies EDT in summer, EST in winter, automatically
    • Correct for any recurring or future-dated event

Command

python
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo

dt = datetime(2026, 8, 21, 14, 30, tzinfo=ZoneInfo("Europe/London"))
dt + timedelta(hours=3)

ZoneInfo("Region/City") tracks real DST rules for that zone; a fixed-offset timezone does not — use ZoneInfo for anything recurring or future-dated.

6.asyncio

asyncio runs many coroutines concurrently on a single thread — an async def function pauses at each await, letting other coroutines run while it waits on I/O instead of blocking the whole program.

Key features

  • async def declares a coroutine function; calling it returns an object, not a result
  • await suspends the current coroutine at a waiting point, letting others run meanwhile
  • asyncio.run(coro()) starts the event loop, runs one coroutine to completion
  • asyncio.gather(*coros) runs several coroutines concurrently and waits for all of them
  • asyncio helps I/O-bound waiting (network, disk); it does not speed up CPU-bound work

asyncio — the surface worth knowing to start

asyncio — the surface worth knowing to start
NameWhat it does
async def f(): ...declares a coroutine function — calling it returns a coroutine object
await exprsuspends the current coroutine until expr completes, yielding control meanwhile
asyncio.run(coro())creates an event loop, runs one coroutine to completion, then closes the loop
asyncio.gather(*coros)runs several coroutines concurrently, returns all results once every one finishes
asyncio.sleep(seconds)a non-blocking wait — yields control instead of freezing the whole program
asyncio.create_task(coro())schedules a coroutine to run concurrently, without waiting for it immediately
Waiting for two slow calls

Sequential (blocking)

  • result_a = fetch(a) # waits fully
  • result_b = fetch(b) # waits fully, after a
  • Total time: time(a) + time(b)

asyncio.gather (concurrent)

  • await asyncio.gather(fetch(a), fetch(b))
  • Both coroutines run, switching at each await
  • Total time: roughly max(time(a), time(b))
  • Sequential (blocking)
    • result_a = fetch(a) # waits fully
    • result_b = fetch(b) # waits fully, after a
    • Total time: time(a) + time(b)
  • asyncio.gather (concurrent)
    • await asyncio.gather(fetch(a), fetch(b))
    • Both coroutines run, switching at each await
    • Total time: roughly max(time(a), time(b))

Command

python
import asyncio

async def main():
    result = await some_coroutine()
    return result

asyncio.run(main())

await only yields control at an actual waiting point — asyncio speeds up I/O-bound waiting, never CPU-bound computation.

7.concurrent.futures

concurrent.futures runs work across a pool of threads or processes behind one shared API — .submit() returns a Future you can .result() later, or .map() runs a function over many inputs at once.

Key features

  • ThreadPoolExecutor(max_workers=N) — threads share memory; good for I/O-bound waiting
  • ProcessPoolExecutor(max_workers=N) — separate processes; CPU-bound work, bypasses the GIL
  • .submit(fn, *args) returns a Future immediately; .result() blocks until it is done
  • .map(fn, iterable) applies fn across many inputs, returning results in order
  • Both work as a context manager — with ...Executor() as ex: waits for all tasks to finish

concurrent.futures — the surface worth knowing

concurrent.futures — the surface worth knowing
NameWhat it does
ThreadPoolExecutor(max_workers=)a pool of worker threads — best for I/O-bound tasks (network, disk)
ProcessPoolExecutor(max_workers=)a pool of worker processes — best for CPU-bound tasks, real parallelism
.submit(fn, *args)schedules one call, returns a Future immediately without blocking
.map(fn, iterable)schedules fn over every item, returns results in the same order as the input
future.result(timeout=None)blocks until the Future finishes, then returns its value (or re-raises its exception)
as_completed(futures)yields each Future as it finishes, not in submission order
Choosing the pool

ThreadPoolExecutor

  • Threads share one process's memory
  • Best for I/O-bound: network calls, file/disk waits
  • GIL still limits true CPU parallelism

ProcessPoolExecutor

  • Each worker is a separate process, own memory
  • Best for CPU-bound: heavy computation
  • Bypasses the GIL — real parallel execution
  • ThreadPoolExecutor
    • Threads share one process's memory
    • Best for I/O-bound: network calls, file/disk waits
    • GIL still limits true CPU parallelism
  • ProcessPoolExecutor
    • Each worker is a separate process, own memory
    • Best for CPU-bound: heavy computation
    • Bypasses the GIL — real parallel execution

Command

python
import concurrent.futures

with concurrent.futures.ThreadPoolExecutor(max_workers=4) as pool:
    results = list(pool.map(fn, items))

ThreadPoolExecutor for I/O-bound waiting, ProcessPoolExecutor for CPU-bound work — submit everything before calling .result() on any of it.

8.logging

logging records leveled, structured messages — DEBUG through CRITICAL — that can be filtered, formatted, and routed to a file or console, unlike print() which always does exactly one thing.

Key features

  • getLogger(name) returns a named logger, usually __name__ — loggers form a name hierarchy
  • Five levels, ascending: DEBUG, INFO, WARNING, ERROR, CRITICAL
  • setLevel() sets the minimum severity that passes through — lower levels are dropped
  • A Handler decides WHERE messages go (console, file); a Formatter decides their shape
  • logger.warning("low stock sku=%s", sku) defers string formatting until actually needed

logging — the core pieces

logging — the core pieces
NameWhat it does
getLogger(name)returns (or creates) a named logger; use __name__ per module
.debug/.info/.warning/.error/.critical(msg, *args)log at each severity level, ascending
.setLevel(level)the minimum severity this logger/handler lets through
StreamHandler / FileHandlersends formatted records to the console / a file
Formatter(fmt)controls the text shape — e.g. "%(levelname)s:%(name)s:%(message)s"
basicConfig(level=, format=)one-call setup for simple scripts — configures the root logger
A message's trip through logging

logger.warning(msg, *args)

called in application code, checked against the logger's level

Logger

has a name and a level; decides whether the message proceeds at all

Handler

has its own level; decides WHERE the message goes (console, file, network)

Formatter

decides the final text shape before it is written

  1. logger.warning(msg, *args) — called in application code, checked against the logger's level
  2. Logger — has a name and a level; decides whether the message proceeds at all
  3. Handler — has its own level; decides WHERE the message goes (console, file, network)
  4. Formatter — decides the final text shape before it is written

Command

python
import logging

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
logger.info("processing order %s", order_id)

Pass %s placeholders and arguments separately to a logging call — the message is only built if it is actually emitted, unlike print() or an f-string.

9.re (regular expressions)

re matches text against a pattern — search() finds the first match anywhere, match() only at the start, findall() gets every match, and sub() replaces matches with new text.

Key features

  • re.search(pattern, s) finds the first match anywhere in s; re.match() only at the start
  • re.findall(pattern, s) returns every non-overlapping match as a list
  • re.sub(pattern, repl, s) replaces every match with repl
  • Parentheses (...) capture a group; m.group(1) reads the first captured piece
  • Use a raw string r"..." for patterns — avoids double-escaping backslashes

re — the functions and syntax worth knowing

re — the functions and syntax worth knowing
NameWhat it does
re.search(pattern, s)returns a Match for the first occurrence anywhere, or None
re.match(pattern, s)like search, but only matches at the very start of s
re.findall(pattern, s)returns every match as a list of strings (or tuples, with groups)
re.sub(pattern, repl, s)returns s with every match replaced by repl
re.compile(pattern)precompiles a pattern for reuse — faster when matched repeatedly
\d \w \s . * + ? {n,m}digit / word char / whitespace / any char / 0+ / 1+ / 0-1 / n-to-m repeats
(...) groups; m.group(n)captures part of the match, readable by position after a match
Reading a match, group by group

re.search(r"(\d{3})-(\d{4})", "call 555-1234 now")

re.search

searches anywhere — unlike re.match(), which anchors to index 0

(\d{3})

group(1) — first parenthesized capture — "555"

(\d{4})

group(2) — second parenthesized capture — "1234"

"call 555-1234 now"

subject text — the raw r"..." prefix passes backslashes through unchanged to this pattern

  • Whole: re.search(r"(\d{3})-(\d{4})", "call 555-1234 now")
  • re.search — searches anywhere: unlike re.match(), which anchors to index 0
  • (\d{3}) — group(1): first parenthesized capture — "555"
  • (\d{4}) — group(2): second parenthesized capture — "1234"
  • "call 555-1234 now" — subject text: the raw r"..." prefix passes backslashes through unchanged to this pattern

Command

python
import re

match = re.search(r"pattern", text)
if match:
    print(match.group())

Use a raw string r"..." for every pattern; re.search() looks anywhere, re.match() only at the start.

10.json

json.dumps() converts a Python dict/list into a JSON string; json.loads() parses a JSON string back into Python objects — the standard round trip for APIs and config files.

Key features

  • json.dumps(obj) returns a JSON string; json.loads(s) parses one back
  • json.dump(obj, file) / json.load(file) do the same directly against a file object
  • dict → object, list → array, str → string, True/False/None → true/false/null
  • indent=2 pretty-prints; sort_keys=True makes output deterministic for diffs/tests
  • A Python-only type (datetime, set, custom class) needs a converter to be JSON-serializable

json — the functions and type mapping worth knowing

json — the functions and type mapping worth knowing
NameWhat it does
json.dumps(obj, indent=, sort_keys=)serializes a Python object to a JSON string
json.loads(s)parses a JSON string into Python objects
json.dump(obj, fp) / json.load(fp)same as dumps/loads, reading/writing a file object directly
dict ↔ objectJSON objects become Python dicts and back
list ↔ array, str ↔ stringstraightforward one-to-one mapping
True/False/None ↔ true/false/nullPython's booleans and None map to JSON's lowercase equivalents

Command

python
import json

text = json.dumps(data, indent=2)
parsed = json.loads(text)

json.dumps/loads round-trip dicts, lists, strings, numbers, booleans, and None exactly — anything else needs a default= converter.

11.enum

enum.Enum defines a fixed, named set of related values as a real type — Status.PENDING is comparable, iterable, and unambiguous, unlike loose string constants like "pending".

Key features

  • class Status(Enum): PENDING = "pending" defines named members with attached values
  • Status.PENDING.name is "PENDING"; Status.PENDING.value is "pending"
  • enum.auto() assigns increasing integers automatically when the value itself doesn't matter
  • list(Status) iterates every member in definition order
  • IntFlag members combine with | for bitmask-style flags (e.g. permissions)

enum — the surface worth knowing

enum — the surface worth knowing
NameWhat it does
class X(Enum): NAME = valuedefines a fixed set of named members, each with an attached value
.name / .valuethe member's identifier string / its attached value
auto()assigns the next integer automatically — use when the value itself is arbitrary
list(X) / X.NAMEiterate every member / access one by its name
X(value)looks up the member whose .value matches, e.g. Status("pending")
IntFlaga variant whose members combine with | into bitmask-style flag sets

Command

python
from enum import Enum

class Status(Enum):
    PENDING = "pending"
    SHIPPED = "shipped"

Compare Enum members to each other (Status.PENDING), not to their raw .value — the member is the identity, the value is just its payload.

12.System interaction: os, sys, subprocess

os reads environment variables and does OS-level file operations, sys exposes interpreter state like command-line arguments, and subprocess runs other programs and captures their output.

Key features

  • os.environ reads environment variables; os.environ.get("KEY", default) avoids a KeyError
  • os.path.join()/os.getcwd() predate pathlib — prefer pathlib.Path for new code
  • sys.argv is the command-line arguments list; sys.argv[0] is the script name itself
  • sys.exit(code) stops the program with an exit code; 0 means success by convention
  • subprocess.run([...], capture_output=True, text=True) captures a program's stdout/stderr

os, sys, and subprocess — the surface worth knowing

os, sys, and subprocess — the surface worth knowing
MemberWhat it does
os.environ / os.environ.get(k, default)reads environment variables, with or without a KeyError on a missing key
os.getcwd() / os.path.join(a, b)the current working directory / joins path segments (pathlib.Path is preferred now)
sys.argvthe list of command-line arguments; index 0 is the script name
sys.exit(code)stops the process immediately with the given exit code
sys.paththe list of directories Python searches when importing a module
subprocess.run(cmd, capture_output=True, text=True)runs an external command, waits, and returns a CompletedProcess with .stdout/.returncode

Command

python
import subprocess

result = subprocess.run(["ls", "-la"], capture_output=True, text=True)
print(result.stdout, result.returncode)

subprocess.run() takes a list of args, not a shell string, unless shell=True is deliberate; os.environ.get() avoids crashing on a missing variable.

Remember

  • Counter tallies, defaultdict auto-creates, namedtuple names fields, deque is fast at both ends.
  • groupby only groups items already adjacent — sort first if matches are scattered through the input.
  • @lru_cache needs hashable arguments; @wraps belongs on every decorator's inner function, without exception.
  • ZoneInfo tracks real DST rules; a fixed-offset timezone does not — use it for anything recurring or future-dated.

Pro tip

  • await only yields control at an actual waiting point — asyncio speeds up I/O waiting, never CPU-bound work.
  • ThreadPoolExecutor for I/O-bound waiting, ProcessPoolExecutor for CPU-bound — submit everything before calling .result().
  • Pass %s placeholders and arguments separately to a logging call — the message is only built if it is actually emitted.
  • Path("a") / "b" joins segments correctly on every OS; never hand-build a path with string concatenation.

Elsewhere in this course

  • contextlib — see Context Managers for the full @contextmanager treatment
  • dataclasses — see Dataclasses and Data Modeling for the full decorator
  • abc / typing — see Abstract classes and interfaces, and Type Hints and Static Typing

Sources

  • The Python Standard Library — docs.python.org/3/library/
  • Every example executed on Python 3.14.3 · checked 2026-08-21

Spotted a mistake on this page? Tell us.