Production error handling

Key features

• A boundary sits at a system edge — an API handler, a queue consumer, a CLI entry point
• Code beneath the boundary raises specific, meaningful exceptions freely
• The boundary catches broadly and translates into one stable response shape
• Without a boundary, internal exception types and details leak to external callers
• Only one boundary is needed per entry point — do not scatter broad catches throughout

Command

def api_boundary(payload):
    try:
        return call_external_api(payload)
    except ValueError as e:
        raise ExternalAPIError(f"upstream call failed: {e}") from e

Key features

• Propagation is automatic — a function with no try/except just lets a risky call propagate
• Each calling function up the stack gets a chance to catch it before the next one
• An exception reaching the top of the stack with no handler crashes the program
• A function need not know or predict what its callers will do with a propagated exception
• Catching too early, deep in a call chain, is often what propagating is the alternative to

Command

def parse_row(row):
    return int(row)   # no try/except — ValueError propagates to the caller

def parse_all(rows):
    return [parse_row(r) for r in rows]   # also does not catch — propagates further

Key features

• The condition causing the failure is expected to change on its own, without code changes
• ConnectionError, TimeoutError, and a 429/503 HTTP response are classic retryable cases
• Retrying only makes sense if the operation is safe to repeat — see idempotency
• A fixed set of retryable types should be defined once, not guessed per call site
• Retrying blindly on every exception type risks retrying a non-retryable error too

Command

RETRYABLE = (ConnectionError, TimeoutError)

def classify(exc):
    return "retry" if isinstance(exc, RETRYABLE) else "fail-fast"

Key features

• The cause is the request itself — bad input, invalid credentials, a malformed query
• ValueError, a 400/401/403/404 HTTP response are classic non-retryable cases
• Retrying wastes time and resources without any chance of a different outcome
• Fail fast instead: surface the error immediately rather than delaying with a retry loop
• Both retryable and non-retryable sets should be explicit, not inferred from any exception

Command

NON_RETRYABLE = (ValueError, InvalidRequestError)

def classify(exc):
    if isinstance(exc, NON_RETRYABLE):
        return "fail-fast"   # retrying the same call cannot help

Key features

• Without a timeout, a hung call can block a program indefinitely
• A timeout converts "unknown, unbounded wait" into a definite TimeoutError
• requests, socket, and most database clients accept a timeout= parameter directly
• A timeout is a classic retryable error — the call might succeed within budget next time
• Set timeouts on every external call by default, not only after a real incident

Command

import requests

try:
    response = requests.get(url, timeout=5)   # raises requests.Timeout past 5 seconds
except requests.Timeout:
    handle_timeout()

Key features

• A retry loop needs a hard cap on attempts — never an unbounded loop
• Only retry exceptions already classified as retryable — see retryable errors
• The last failure should be re-raised, not silently discarded, once attempts run out
• A fixed delay is the simplest strategy; exponential backoff improves on it
• Retrying only makes sense for an operation safe to repeat (idempotent)

Command

def retry(fn, attempts, delay):
    last_exc = None
    for attempt in range(1, attempts + 1):
        try:
            return fn()
        except RETRYABLE as e:
            last_exc = e
            if attempt < attempts:
                time.sleep(delay)
    raise last_exc

Key features

• delay = base * (2 ** (attempt-1)), capped — 1, 2, 4, 8, 16, 30, 30 for base=1, cap=30
• The cap matters — uncapped doubling reaches minutes after only a handful of attempts
• Jitter (randomizing within the delay) avoids many clients retrying in lockstep at once
• Backoff only helps for retryable errors — see retryable errors
• Combine with a capped attempt count — backoff alone does not bound total retries

Command

def compute_backoff(base, attempt, cap):
    delay = base * (2 ** (attempt - 1))   # attempt is 1-indexed
    return min(delay, cap)

Key features

• logger.exception(msg) only makes sense inside except — it reads the active exception
• It logs at ERROR level and appends the full traceback automatically
• Equivalent to logger.error(msg, exc_info=True), just shorter
• print(e) loses the traceback entirely — only the message string survives
• Log once, where the exception is handled — not at every level it merely passes through

Command

try:
    risky_operation()
except ValueError:
    logger.exception("risky_operation failed")   # message + full traceback

Key features

• Internal (logged): full exception, stack trace, ids, table names — as detailed as needed
• External (returned/shown): short, generic, honest — no internals, no stack trace
• Exposing internal detail externally is an information-leak risk, not just an aesthetic one
• The two messages are built at the exception boundary — see exception boundaries
• A generic external message still needs to be genuinely helpful, not just vague

Command

except OrderError as e:
    logger.error("order failed: %s", e)                # detailed, internal
    return {"ok": False, "error": "We could not process your order."}  # generic, external

Key features

• except Exception: pass discards the exception with zero trace — no log, no re-raise
• The operation LOOKS like it succeeded to anything watching from outside
• At minimum, log the exception even if the code truly cannot handle it usefully
• A narrower except type plus a comment explaining WHY is the justified exception
• Swallowing differs from handling — handling responds to a failure; swallowing does nothing

Command

try:
    risky_call()
except Exception:
    pass   # ANTI-PATTERN — no log, no metric, no trace this ever happened