beginnerintermediateadvanced

Python · Exception Handling

Production error handling

Concepts: 10

Where to catch exceptions in a real system, which failures are worth retrying and how to space out those retries, and the difference between what gets logged and what a caller is actually allowed to see.

This section

#63 Python Tutorial for Beginners | Exception HandlingTelusko

Boundaries and propagation

Where exceptions are meant to be caught in a call chain, and what happens by default when nothing catches them.

Exception boundaries

standardintermediate

An exception boundary is the one deliberate place — an API handler, a job entry point — where every exception from the code it calls is caught and translated into one stable, intentional response, instead of leaking internal detail outward.

Think of it as

A boundary is a translation desk at a border crossing, not a wall stopping everything — code beneath it can raise whatever specific exceptions make sense internally; the boundary is the one place that converts all of them into a small, stable set of responses the outside world actually understands.

python

def api_boundary(payload):
    try:
        return call_external_api(payload)
    except ValueError as e:
        raise ExternalAPIError(f"upstream call failed: {e}") from e

What we're doing: Wrap a call that can raise an internal ValueError with a boundary that translates it into one stable ExternalAPIError, preserving the original as the cause.

boundary.pypython

class ExternalAPIError(Exception):
    pass


def call_external_api(payload):
    if payload is None:
        raise ValueError("payload required")
    return {"status": "ok"}


def api_boundary(payload):
    try:
        return call_external_api(payload)
    except ValueError as e:
        raise ExternalAPIError(f"upstream call failed: {e}") from e


try:
    api_boundary(None)
except ExternalAPIError as e:
    print(f"boundary caught: {e}")

5: call_external_api raises whatever specific exception makes sense internally — here, a plain ValueError.
11: api_boundary is the one place that catches it and translates it into one stable ExternalAPIError.
12: from e preserves the original as __cause__, so nothing is lost — only the outward-facing type changes.

Output

boundary caught: upstream call failed: payload required

Why this works: call_external_api is free to raise ValueError, TypeError, or any other exception that fits its own logic — api_boundary is the single place that catches those and re-raises one stable ExternalAPIError, so every caller of api_boundary only ever needs to handle one exception type, regardless of how many different failures can happen underneath it.

Remember: Put one broad catch at the edge of a system — not scattered through internal code — and translate everything into a stable, intentional response there.

Error propagation

standardintermediate

An exception with no matching except in the current function keeps traveling up the call stack, unchanged, to whichever caller has one — or crashes the program if none does. This is the default behaviour.

Think of it as

Propagation is gravity, not a feature you turn on — an unhandled exception simply keeps falling upward through each calling function until something catches it, exactly the way a dropped exception object needs no help to keep going.

python

def parse_row(row):
    return int(row)   # no try/except — ValueError propagates to the caller

def parse_all(rows):
    return [parse_row(r) for r in rows]   # also does not catch — propagates further

What we're doing: Show a ValueError raised deep inside a list comprehension propagate untouched through two function calls to the code that finally catches it.

propagate.pypython

def parse_row(row):
    return int(row)

def parse_all(rows):
    return [parse_row(r) for r in rows]

try:
    parse_all(["1", "2", "x"])
except ValueError as e:
    print(f"propagated to caller: {e}")

2: int(row) raises ValueError for "x" — parse_row has no try/except, so it does not catch it.
5: parse_all calls parse_row inside a comprehension and also has no try/except — the exception keeps going.
9: The first actual except in the whole call chain is here, two frames up from where int() raised.

Output

propagated to caller: invalid literal for int() with base 10: 'x'

Why this works: Neither parse_row nor parse_all contains a try/except, so Python does not stop at either of them — the ValueError keeps unwinding the call stack, frame by frame, until it reaches the try in the caller, which is the first place actually equipped to handle it.

Remember: An unhandled exception propagates automatically up the call stack. Catch where you can actually respond, not reflexively at every level.

Deciding what to retry

Transient failures worth a second attempt, request failures that are not, and bounding how long any single attempt is allowed to take.

Retryable errors

standardintermediate

A retryable error comes from a transient condition — a dropped connection, a rate limit, a momentary timeout — where trying the exact same call again has a real chance of succeeding.

Think of it as

Retryable means "the problem was with the moment, not the request" — the same network call that failed because a server was briefly overloaded can succeed a second later with identical arguments, which is what makes retrying worth doing at all.

python

RETRYABLE = (ConnectionError, TimeoutError)

def classify(exc):
    return "retry" if isinstance(exc, RETRYABLE) else "fail-fast"

What we're doing: Define a fixed tuple of retryable exception types and classify three different failures against it.

classify.pypython

class RateLimitError(Exception):
    """Retryable — the server asked us to slow down."""


class InvalidRequestError(Exception):
    """Non-retryable — the request itself is wrong; retrying repeats the failure."""


RETRYABLE = (RateLimitError, TimeoutError, ConnectionError)


def classify(exc):
    if isinstance(exc, RETRYABLE):
        return "retry"
    return "fail-fast"


print(classify(RateLimitError()))
print(classify(TimeoutError()))
print(classify(InvalidRequestError()))

9: RETRYABLE is defined once as a fixed tuple, reused for every classification — not re-decided at each call site.
13: isinstance(exc, RETRYABLE) checks against the whole tuple in one call, the same mechanism except (A, B): uses.

Output

retry
retry
fail-fast

Why this works: RateLimitError and TimeoutError both represent conditions expected to pass on their own — a rate limit resets, a slow response might succeed on a second attempt — so both classify as "retry." InvalidRequestError means the request itself is malformed, which retrying cannot fix, so it classifies as "fail-fast" instead.

Remember: A retryable error comes from a transient condition expected to change. Define the retryable set once, as a fixed tuple.

Non-retryable errors

standardintermediate

A non-retryable error is caused by the request itself, not the moment — bad input, a missing permission, a malformed URL. Retrying the identical call cannot succeed, because nothing about the underlying problem changes on its own.

Think of it as

Non-retryable means "the moment has nothing to do with it" — mailing the same incorrectly-addressed letter a second time does not deliver it; the address has to change, not the timing, and only new code (fixing the request) can fix a non-retryable error.

python

NON_RETRYABLE = (ValueError, InvalidRequestError)

def classify(exc):
    if isinstance(exc, NON_RETRYABLE):
        return "fail-fast"   # retrying the same call cannot help

What we're doing: Classify a non-retryable InvalidRequestError against a fixed set and confirm it is routed to fail fast rather than retried.

fail_fast.pypython

class InvalidRequestError(Exception):
    """Non-retryable — the request itself is wrong; retrying repeats the failure."""


NON_RETRYABLE = (InvalidRequestError, ValueError)


def classify(exc):
    if isinstance(exc, NON_RETRYABLE):
        return "fail-fast"
    return "retry"


try:
    raise InvalidRequestError("missing required field: email")
except InvalidRequestError as e:
    print(f"{classify(e)}: {e}")

5: NON_RETRYABLE is a fixed tuple, defined once, the same pattern retryable-errors uses for its own set.
9: A request-shaped failure — a missing field — is routed to fail-fast, not into a retry loop.

Output

fail-fast: missing required field: email

Why this works: A missing required field is a property of this specific request — sending the identical request again reproduces the identical error every time, so classifying it as "retry" would only add delay before the same failure is reported. Failing fast surfaces the real problem immediately instead.

Remember: A non-retryable error comes from the request itself — retrying the identical call cannot succeed. Fail fast instead of looping.

Timeouts

standardintermediate

A timeout bounds how long an operation is allowed to run before it is treated as failed and TimeoutError is raised — without one, a caller can wait forever on a network call, lock, or process that never returns.

Think of it as

A timeout is a deadline the caller sets, not the callee — the code being called usually has no idea it is running long; the caller decides how much waiting is acceptable and enforces that limit itself, converting "still running" past that point into a definite failure.

python

import requests

try:
    response = requests.get(url, timeout=5)   # raises requests.Timeout past 5 seconds
except requests.Timeout:
    handle_timeout()

What we're doing: Wrap a call with a time budget and raise TimeoutError when it runs past that budget, using only the standard library so the check is runnable without a network dependency.

deadline.pypython

import time


def call_with_deadline(fn, seconds):
    start = time.monotonic()
    result = fn()
    elapsed = time.monotonic() - start
    if elapsed > seconds:
        raise TimeoutError(f"call took {elapsed:.3f}s, exceeded {seconds}s budget")
    return result


print(call_with_deadline(lambda: 1 + 1, seconds=1.0))

try:
    call_with_deadline(lambda: time.sleep(0.05) or "done", seconds=0.01)
except TimeoutError as e:
    print(f"timeout: {e}")

5: time.monotonic() measures elapsed wall-clock time, unaffected by system clock adjustments.
8: Exceeding the budget raises TimeoutError after the call already finished — a real client library enforces the limit while the call is still running.

Output

2
timeout: call took 0.050s, exceeded 0.01s budget

Why this works: This simplified version measures elapsed time after the call returns, purely to demonstrate the comparison in a runnable, dependency-free example — real HTTP and database clients enforce the limit while the call is in flight, using a timeout= parameter, and interrupt the call itself rather than waiting for it to finish first.

Remember: Set a timeout on every external call — without one, a hung dependency can block your program indefinitely instead of failing predictably.

How to retry

The shape of a retry loop, and spacing retries out so they give a struggling dependency room to recover.

Retry strategies

standardintermediate

A retry strategy is a loop with a capped number of attempts, a delay between them, and a final re-raise if every attempt fails — never an unbounded while True around a call that keeps failing.

Think of it as

A retry loop is a countdown, not a promise — it gives an operation a fixed number of extra chances, then gives up and reports failure exactly like it would have on attempt one, so a caller can never be surprised by silent infinite retrying.

python

def retry(fn, attempts, delay):
    last_exc = None
    for attempt in range(1, attempts + 1):
        try:
            return fn()
        except RETRYABLE as e:
            last_exc = e
            if attempt < attempts:
                time.sleep(delay)
    raise last_exc

What we're doing: Retry a function that fails twice before succeeding, using a capped attempt count, and confirm the final result and the number of attempts used.

retry_fixed.pypython

import time


def flaky(attempt_counter):
    attempt_counter[0] += 1
    if attempt_counter[0] < 3:
        raise ConnectionError("connection refused")
    return "connected"


def retry_fixed(fn, attempts, delay):
    last_exc = None
    for attempt in range(1, attempts + 1):
        try:
            return fn()
        except ConnectionError as e:
            last_exc = e
            print(f"attempt {attempt} failed: {e}")
            if attempt < attempts:
                time.sleep(delay)
    raise last_exc


counter = [0]
result = retry_fixed(lambda: flaky(counter), attempts=5, delay=0)
print(f"result: {result}, attempts used: {counter[0]}")

11: attempts=5 is the hard cap — the loop can never run more than 5 times, however many failures happen.
20: raise last_exc only runs if every attempt failed — the real failure is never silently discarded.

Output

attempt 1 failed: connection refused
attempt 2 failed: connection refused
result: connected, attempts used: 3

Why this works: flaky() fails on its first two calls and succeeds on the third, so retry_fixed returns as soon as fn() succeeds — attempt 3 never reaches the except block at all. Had every one of the 5 attempts failed instead, raise last_exc would re-raise the final ConnectionError exactly as if no retry loop had been used, rather than swallowing the failure.

Remember: Cap the attempt count, retry only classified-retryable exceptions, and re-raise the final failure once exhausted.

Exponential backoff

coreintermediate

Exponential backoff doubles the delay before each retry — 1s, 2s, 4s, 8s — capped at a maximum, so retries spread out over time instead of hammering a struggling dependency at a constant rate.

Think of it as

Backoff is giving a struggling system room to breathe, not just waiting longer for no reason — a constant one-second retry keeps hitting a server at the same rate that may already be the problem; doubling the wait each time gives it steadily more time to recover, and the cap stops the wait from growing forever.

python

def compute_backoff(base, attempt, cap):
    delay = base * (2 ** (attempt - 1))   # attempt is 1-indexed
    return min(delay, cap)

What we're doing: Compute real backoff delays for seven attempts with a 1-second base and 30-second cap, then add random jitter so retries do not all land at the exact same moment.

backoff.pypython

import random


def compute_backoff(base, attempt, cap):
    delay = base * (2 ** (attempt - 1))
    return min(delay, cap)


base_delay = 1.0
cap = 30.0
delays = [compute_backoff(base_delay, n, cap) for n in range(1, 8)]
print(delays)

random.seed(42)
jittered = [
    round(random.uniform(0, compute_backoff(base_delay, n, cap)), 4)
    for n in range(1, 6)
]
print(jittered)

5: 2 ** (attempt - 1) doubles the raw delay every attempt: 1, 2, 4, 8, 16, 32, 64 before the cap is applied.
6: min(delay, cap) is what turns 32 and 64 into 30 — the cap stops unbounded growth.
16: random.uniform(0, delay) is "full jitter" — a random point between zero and the computed delay, not the full delay every time.

Output

[1.0, 2.0, 4.0, 8.0, 16.0, 30.0, 30.0]
[0.6394, 0.05, 1.1001, 1.7857, 11.7835]

Why this works: The uncapped sequence would be 1, 2, 4, 8, 16, 32, 64 — doubling every attempt — but min(delay, cap) flattens attempts 6 and 7 to 30.0, so the wait stops growing past the cap instead of reaching over a minute. The jittered sequence is different every run without a fixed seed — random.uniform(0, delay) picks a random point under each capped delay, which is what prevents many clients retrying after an outage from all resuming at the exact same instant.

Delay doubles each retry, then caps

1s → 2s → 4s

delay doubles after each failed attempt

8s → 16s

keeps doubling while under the cap

30s (capped)

stops growing once it reaches the maximum

Doubling the delay with no cap at all

Wrong

python

def compute_backoff(base, attempt):
    return base * (2 ** (attempt - 1))   # WRONG — no cap

# after 10 failed attempts: base=1 gives a 512-second (8.5 minute) wait

Better

python

def compute_backoff(base, attempt, cap):
    delay = base * (2 ** (attempt - 1))
    return min(delay, cap)   # never waits longer than cap

What you see: A job that fails repeatedly ends up waiting many minutes between attempts, even though the dependency may have recovered long before — the exponential growth has nothing stopping it.

Why: base * (2 ** (attempt - 1)) grows without bound as attempts increase — by attempt 10 with base=1 the raw delay is 512 seconds. A cap keeps the wait within a sane, bounded range no matter how many attempts are made.

Backoff delay by attempt — base=1.0s, cap=30.0s

Backoff delay by attempt — base=1.0s, cap=30.0s
Attempt	Uncapped (base × 2^(n−1))	Actual delay (capped)
1	1.0	1.0
2	2.0	2.0
3	4.0	4.0
4	8.0	8.0
5	16.0	16.0
6	32.0	30.0
7	64.0	30.0

Together

python

def compute_backoff(base, attempt, cap):
    delay = base * (2 ** (attempt - 1))
    return min(delay, cap)

delays = [compute_backoff(1.0, n, 30.0) for n in range(1, 8)]
print(delays)

Remember: Double the delay each retry — base × 2^(attempt−1) — and always cap it. Add jitter so clients do not all resume at once.

See also: retry strategies · retryable errors · timeouts

Reporting failures honestly

Recording enough detail to debug a failure later, showing a caller only what they need, and never discarding a failure with no trace at all.

Logging exceptions

standardintermediate

logger.exception(message), called from inside an except block, logs the message at ERROR level and attaches the full traceback automatically — the same as logger.error(message, exc_info=True), but shorter to write.

Think of it as

logger.exception() is logger.error() that remembers where it is standing — called inside an except block, it can see the exception currently being handled and attaches its full traceback without you passing it explicitly.

python

try:
    risky_operation()
except ValueError:
    logger.exception("risky_operation failed")   # message + full traceback

What we're doing: Log a caught exception with logger.exception() and confirm the output includes both the message and the full traceback text.

log_exception.pypython

import logging

logger = logging.getLogger(__name__)


def risky_operation():
    raise ValueError("bad state")


try:
    risky_operation()
except ValueError:
    logger.exception("risky_operation failed")

6: risky_operation raises ValueError("bad state") — the exception being handled when except runs.
13: logger.exception reads that active exception automatically — no need to pass it as an argument.

Output

ERROR:__main__:risky_operation failed
Traceback (most recent call last):
  File "log_exception.py", line 11, in <module>
    risky_operation()
  File "log_exception.py", line 7, in risky_operation
    raise ValueError("bad state")
ValueError: bad state

Why this works: logger.exception() calls logger.error() internally with exc_info=True already set, and exc_info=True tells the logging module to look up sys.exc_info() — the exception currently propagating through the active except block — and format its full traceback into the log record, without the caller needing to pass the exception object explicitly.

Logging with print(e) or logger.error(e), losing the traceback

Wrong

python

try:
    risky_operation()
except ValueError as e:
    logger.error(f"failed: {e}")   # message only — traceback is gone

Better

python

try:
    risky_operation()
except ValueError:
    logger.exception("risky_operation failed")   # message + full traceback

What you see: The log shows "failed: bad state" but nothing about which line raised it or what called that function — reproducing the bug from the log alone is much harder.

Why: str(e) only ever returns the exception's message string — it carries none of the traceback, so logger.error(f"failed: {e}") discards exactly the information (file, line, call chain) most useful for debugging a production failure after the fact.

Remember: Call logger.exception(message) from inside except to log the message and the full traceback together — f"{e}" alone throws the traceback away.

User-facing vs internal errors

standardintermediate

Log the full, detailed exception internally — stack trace, table names, ids — and return a short, generic message to the caller. The same detail exposed externally leaks implementation.

Think of it as

It is the difference between a doctor's chart and what they tell the patient — the chart records every specific, technical detail for whoever treats the case next; the patient hears a clear, honest, but much shorter summary that does not require a medical degree to understand.

python

except OrderError as e:
    logger.error("order failed: %s", e)                # detailed, internal
    return {"ok": False, "error": "We could not process your order."}  # generic, external

What we're doing: Catch a detailed internal exception at an API boundary, log the full detail, and return a short generic message to the caller instead.

boundary_errors.pypython

import logging

logger = logging.getLogger(__name__)


class OrderError(Exception):
    """Internal, detailed — for logs."""


def place_order(item_id, stock):
    if item_id not in stock:
        raise OrderError(
            f"item_id={item_id} missing from inventory table 'stock_2026' row count={len(stock)}"
        )
    return "ordered"


def place_order_endpoint(item_id, stock):
    try:
        return {"ok": True, "result": place_order(item_id, stock)}
    except OrderError as e:
        logger.error("order failed: %s", e)
        return {"ok": False, "error": "We could not process your order. Please try again."}


print(place_order_endpoint("sku-404", {"sku-1": 5}))

12: The internal message names the exact table and row count — useful to an engineer, meaningless (or risky) to a customer.
21: logger.error keeps that full detail, only in the logs — it never reaches the returned response.
22: The returned message is short, generic, and reveals nothing about the database or inventory system.

Output

{'ok': False, 'error': 'We could not process your order. Please try again.'}

Why this works: The detailed OrderError message — mentioning the inventory table name and row count — only ever reaches logger.error, which writes to internal logs an engineer can search. The dict returned to the caller carries a separate, deliberately generic string, so a customer (or an attacker probing the API) learns nothing about the database schema behind the failure.

Remember: Log the full, detailed exception internally; return a short, generic, honest message externally. Detail exposed externally is a leak.

Avoiding swallowed exceptions

coreintermediate

A swallowed exception is caught and discarded with no trace at all — except Exception: pass. The failure still happened, but nothing logs it, so it looks to everyone else like the operation succeeded.

Think of it as

A swallowed exception is a smoke alarm with the battery removed — the fire (the real failure) still happens; the only thing missing is any signal that it did, which means nobody finds out until the damage is much harder to trace back to its cause.

python

try:
    risky_call()
except Exception:
    pass   # ANTI-PATTERN — no log, no metric, no trace this ever happened

What we're doing: Parse a list of rows, showing that silently swallowing per-row failures returns success-looking output identical in shape to a version that logs each failure — the difference only shows up in the logs.

swallow.pypython

import logging

logger = logging.getLogger(__name__)


def swallow_bad(rows):
    results = []
    for r in rows:
        try:
            results.append(int(r))
        except Exception:
            pass  # ANTI-PATTERN — silently drops bad rows, no trace
    return results


def swallow_good(rows):
    results = []
    for r in rows:
        try:
            results.append(int(r))
        except ValueError:
            logger.error("skipping unparseable row: %r", r)
            continue
    return results


print(swallow_bad(["1", "x", "3"]))
print(swallow_good(["1", "x", "3"]))

11: except Exception: pass drops the failing row "x" with no trace it ever failed — results just comes back shorter.
22: except ValueError (narrower) plus a log line keeps the exact same recovery behaviour, but leaves a trace.

Output

[1, 3]
ERROR:__main__:skipping unparseable row: 'x'
[1, 3]

Why this works: Both functions return the identical [1, 3] — from the caller's point of view, the two versions look interchangeable. The only difference is that swallow_good leaves a log line recording that "x" failed to parse; swallow_bad leaves nothing at all, so a caller who expected 3 results back and got 2 has no way to find out why without reading the source code.

Swallowed vs. handled

exception raised

something genuinely went wrong

except Exception: pass

discarded — zero trace anywhere

looks like success

caller, logs, and metrics all see nothing

except Exception: pass — the roadmap's named anti-pattern

Wrong

python

try:
    save_to_database(record)
except Exception:
    pass   # the save silently failed — nobody will ever know

Better

python

try:
    save_to_database(record)
except IntegrityError as e:
    logger.error("failed to save record %s: %s", record.id, e)
    raise   # or handle it deliberately — never just discard it

What you see: A database write silently fails and no error appears anywhere — the bug only surfaces later as "missing data," often far from where the actual failure happened.

Why: except Exception: pass matches every ordinary exception and does nothing with it — not even a log line — so the failure leaves no trace an engineer could search for. This is safe only when the exception is both narrowly typed AND genuinely inconsequential, and even then the roadmap's own rule is that it needs a comment justifying why — a bare pass is never self-explanatory.

Remember: except Exception: pass discards a real failure with zero trace. At minimum, log it, and catch the narrowest type you actually expect.