Lesson 2: Exception Mechanics

try, except, else, finally

Python's exception statement has four parts, and most code uses two. The other two are worth knowing because they express intent that the two-part version cannot.

try:
    response = client.post(url, json=payload)
except httpx.TimeoutException:
    logger.warning("request timed out")
    raise
else:
    return response.json()
finally:
    metrics.record_attempt(url)

try holds the code that might fail. Keep it as small as possible, wrapping only the operation that can actually raise. A large try block catches exceptions from lines you did not intend to guard, which hides real bugs.

# Too broad: a bug in build_payload is silently treated as a network failure
try:
    payload = build_payload(document)
    response = client.post(url, json=payload)
    return parse_response(response)
except httpx.HTTPError:
    return None

# Precise: only the network call is guarded
payload = build_payload(document)
try:
    response = client.post(url, json=payload)
except httpx.HTTPError:
    return None
return parse_response(response)

except catches a specific exception type. You can have several, and they are tried in order, so more specific types must come before more general ones.

try:
    response = client.post(url, json=payload)
except httpx.TimeoutException:
    ...                                    # specific first
except httpx.HTTPError:
    ...                                    # TimeoutException is a subclass of this

Reversing those two means the timeout branch never runs, because the general case catches it first. This produces no warning.

else runs only if no exception was raised. Its value is that it separates "the risky thing worked" from "the risky thing". Code in else is not protected by the except clauses, which is usually what you want.

try:
    response = client.post(url, json=payload)
except httpx.HTTPError as exc:
    raise ProviderError("request failed") from exc
else:
    return parse_response(response)        # a parse failure here is not a network error

If parse_response had been inside the try, a parsing bug would be caught by the except clause and mislabelled as a provider error. The else clause prevents that whole category of confusion.

finally runs no matter what: on success, on exception, and even on return from inside the try. Use it for cleanup that must happen regardless.

def process_batch(batch: list[Document]) -> BatchResult:
    started = time.perf_counter()
    try:
        return call_provider(batch)
    finally:
        metrics.record_duration(time.perf_counter() - started)

The duration is recorded whether the call succeeded or failed, which is the only version that gives you useful latency data, since failures are often the slow ones.

Module 4 used finally for exactly this reason: to guarantee the run report was written whichever way the job ended.

[IMAGE PROMPT M5-2
Purpose: Show which blocks execute in the success case and the exception case, including the fact that finally always runs.
Visual type: Two-path execution flow diagram.
Prompt: A clean educational flow diagram with a single entry point at the top labelled "enter try block", splitting into two labelled vertical paths. The left path is headed "no exception raised" and passes through three stacked boxes in order: "try body completes", "else runs", "finally runs", ending at a terminal labelled "return value". The right path is headed "exception raised" and passes through four stacked boxes in order: "try body stops at the failing line", "matching except runs", "else is skipped", "finally runs", ending at a terminal labelled "return or re-raise". A horizontal dashed line connects the two "finally runs" boxes, annotated "finally runs on both paths, always". A note beside the right path's "else is skipped" box reads "this is why else separates success from risk".
Required elements: One entry point, two clearly headed paths, the stated boxes in order on each path with else skipped on the right, both finally boxes linked by an annotated connector, two terminals, the note.
Style: Clean educational illustration, professional, uncluttered, high contrast, flat vector, consistent box sizing.
Layout: Vertical flow from a single top entry, splitting into two parallel columns, with the dashed connector crossing between them near the bottom.
Text labels: "enter try block", "no exception raised", "exception raised", "try body completes", "try body stops at the failing line", "matching except runs", "else runs", "else is skipped", "finally runs", "return value", "return or re-raise", "finally runs on both paths, always", "this is why else separates success from risk".
Aspect ratio: 4:3
Accessibility: Distinguish the two paths using headers, box text, and position rather than colour alone.
Avoid: Code screenshots, decorative icons, tiny text, logos, watermarks.
Alt text: Flow diagram showing that on the success path the try body, else, and finally all run, while on the exception path the try body stops, a matching except runs, else is skipped, and finally still runs.
END IMAGE PROMPT]

Why bare except is a bug

try:
    response = client.post(url, json=payload)
except:                      # bare except
    return None

This catches everything, and everything includes things you never intended.

It catches KeyboardInterrupt, so pressing Ctrl-C does not stop your program. It catches SystemExit, so a deliberate shutdown is swallowed. It catches MemoryError, at the point where continuing is hopeless. And it catches every bug in your own code, so a typo in a variable name inside the try block becomes a silent return None instead of a NameError that would have told you exactly what was wrong.

That last consequence is the one that costs the most time. A bare except converts loud, informative failures into quiet wrong behaviour, and the symptom appears somewhere else entirely.

except Exception: is better, because KeyboardInterrupt and SystemExit inherit from BaseException rather than Exception and are therefore not caught. It is still too broad for most uses, but it is legitimate in exactly one place: the top level of a long-running job, where you want to log an unexpected failure and continue or shut down cleanly.

for document in documents:
    try:
        process(document)
    except (ProviderError, DataError) as exc:
        record_rejection(document, str(exc))            # expected failures
    except Exception:
        logger.exception("unexpected failure on %s", document.id)
        raise                                           # unexpected: log and stop

Note the raise at the end. Catching a broad exception to log it and then re-raising preserves the failure while ensuring it is recorded. Catching it and continuing silently is what turns a two million document job into a job that produced eleven records and reported success.

The rule. Catch the narrowest exception that describes what you are handling. If you find yourself catching broadly, ask what you would actually do differently for each type, and usually the answer reveals which specific types you meant.

Exception chaining with raise from

When you catch a low-level exception and raise a domain-specific one, the original context matters.

# Loses the original
try:
    response = client.post(url, json=payload)
except httpx.HTTPError:
    raise ProviderError("embedding request failed")

# Preserves it
try:
    response = client.post(url, json=payload)
except httpx.HTTPError as exc:
    raise ProviderError("embedding request failed") from exc

The from exc produces the chained traceback covered in Module 1, joined by the sentence about the above exception being the direct cause. Your reader gets both the meaningful domain error and the technical origin: not just "embedding request failed" but "embedding request failed, because the connection to this host was reset".

Without from, Python still shows both tracebacks, but joined by "during handling of the above exception, another exception occurred", which implies an accident rather than a deliberate translation. Module 1 taught reading that distinction. This is the writing side of it.

Use raise ... from None to deliberately suppress the original when it is noise rather than signal, which is rare and should be a considered choice.

Custom exception hierarchies

Library exceptions describe what went wrong technically. Your code needs exceptions describing what went wrong in your domain, so that callers can respond to categories rather than to implementation details.

class AppError(Exception):
    """Base for every error this application raises deliberately."""


class ValidationError(AppError):
    """Input data did not meet requirements."""


class RetrievalError(AppError):
    """A document or chunk could not be fetched."""


class ProviderError(AppError):
    """A call to an external model or embedding provider failed."""


class RateLimitError(ProviderError):
    """The provider is rate limiting us."""
    def __init__(self, message: str, *, retry_after: float | None = None) -> None:
        super().__init__(message)
        self.retry_after = retry_after


class ProviderTimeoutError(ProviderError):
    """The provider did not respond in time."""


class ProviderOverloadedError(ProviderError):
    """The provider reported that it is overloaded."""


class OutputError(AppError):
    """The provider responded, but the content was unusable."""

The hierarchy is:

AppError
├── ValidationError
├── RetrievalError
├── ProviderError
│   ├── RateLimitError
│   ├── ProviderTimeoutError
│   └── ProviderOverloadedError
└── OutputError

Three things this buys you.

Callers choose their granularity. A caller that wants to handle any provider problem catches ProviderError. A caller that needs to wait a specific time catches RateLimitError and reads retry_after. Neither has to know that the underlying library was httpx, which means swapping the HTTP client later does not change any calling code.

Your errors are distinguishable from bugs. except AppError: catches every failure you anticipated. Anything else reaching the top level is unanticipated, and that distinction is what lets the loop in the previous section handle expected failures gracefully while surfacing real bugs.

Exceptions carry data. RateLimitError holding retry_after is the pattern to copy. An exception is an object, and putting the relevant value on it is far better than parsing it back out of a message string.

Translating at the boundary. The hierarchy is only useful if something converts library exceptions into it, and that conversion belongs in one place:

def call_provider(payload: dict) -> dict:
    """Call the provider, translating transport failures into domain errors."""
    try:
        response = client.post(PROVIDER_URL, json=payload)
    except httpx.TimeoutException as exc:
        raise ProviderTimeoutError("provider did not respond in time") from exc
    except httpx.HTTPError as exc:
        raise ProviderError("transport failure calling provider") from exc

    if response.status_code == 429:
        retry_after = parse_retry_after(response.headers.get("retry-after"))
        raise RateLimitError("rate limited by provider", retry_after=retry_after)
    if response.status_code in (502, 503, 504):
        raise ProviderOverloadedError(f"provider returned {response.status_code}")
    if response.status_code >= 400:
        raise ProviderError(f"provider returned {response.status_code}")

    return response.json()

This is the parse-at-the-boundary principle from Module 2 applied to failures rather than to data. One function knows about httpx and status codes. Everything above it knows only about your domain.