Lesson 1: Failure Taxonomy

The problem this lesson solves

The recipe-extractor pipeline from Module 4 streams, resumes, and reports on itself. It also now calls an external service, because chunks need embeddings and documents need summarising. That single change moves it from a program that either works or has a bug into a program whose correctness depends on a machine you do not own.

Here is a real overnight run. It starts at 22:00. At 23:40 it slows, because the provider is rate limiting and your code is retrying every failure instantly. At 01:15 a request hangs, and because no timeout was set, it hangs forever, holding the pipeline still. At 04:00 the connection resets. The job is discovered at 09:00 having processed 800,000 of two million documents, with no indication of which failures were transient and which were real.

None of that is a bug in the sense of wrong logic. Every line does what it says. The program simply has no answer for the world being unreliable, and this module is about building that answer.

The first step is being able to name what went wrong, because different failures need opposite responses. Retrying a rate limit is correct. Retrying a malformed request is a waste that makes the situation worse.

Seven kinds of failure

Syntax errors are caught before the program runs. Python refuses to execute a file it cannot parse. These do not reach production and are not interesting here, beyond noting that the tooling from Module 1 catches them at the moment you type.

Runtime errors happen while running and come from your code being wrong about something. A TypeError from passing a string where a number was expected, an AttributeError from a None that should have been an object, a KeyError from a key that was never there. They are deterministic: given the same input the same error occurs. They are bugs, and the correct response is to fix the code, not to retry.

Logical errors produce no exception at all. The program runs to completion and gives a wrong answer. Chunks overlap incorrectly, the deduplication misses matches because of the Unicode issue from Module 4, a filter keeps the wrong side. These are the most expensive kind because nothing tells you they happened. Only tests, evaluation, and someone noticing catch them.

Data errors happen when your code is right and the input is not what it promised. A file in an unexpected encoding, a JSON field that is a string where a list was expected, an empty document, a recipe page that is actually a category listing. These are not bugs in your logic, and the correct response is usually neither to crash nor to retry, but to reject the record with a reason, which is exactly what Module 2 built.

Network errors are failures of the connection itself, before any application response exists. DNS resolution fails, the connection is refused, a TLS handshake fails, a connection resets mid-transfer, or the request times out. Most are transient. Retrying is usually correct.

Provider errors are responses from a service that reached you and reported a problem. A 429 saying you are being rate limited, a 500 saying something broke on their side, a 503 saying they are overloaded, a 401 saying your key is wrong, a 400 saying your request was malformed. The status code carries the crucial information: whether waiting will help.

Model errors are specific to systems built on language models and are the category most often missed. The call succeeded, the status was 200, and the response is still unusable. The JSON has a trailing comma. The output was cut off because it hit the token limit. The model returned prose wrapped around the JSON you asked for. The model refused. The output is well-formed and factually wrong.

Model errors deserve emphasis because every other layer reports success. Your HTTP client is satisfied, your retry logic sees no failure, and your monitoring shows a healthy service. The failure is entirely in the content, and only validation at the boundary finds it. Module 2 established that principle, and this is where it earns its keep.

[IMAGE PROMPT M5-1
Purpose: Give learners a reference for the seven failure categories, showing when each is detected and what the correct response is.
Visual type: Categorised reference chart with three annotated columns.
Prompt: A clean educational chart with seven rows and three labelled columns. The column headers read "Failure type", "When you find out", and "Correct response". The seven rows read, in order: "Syntax" with "before running" and "fix the code"; "Runtime" with "during execution, deterministic" and "fix the code"; "Logical" with "never, unless tested" and "tests and evaluation"; "Data" with "at the input boundary" and "reject with a reason"; "Network" with "before a response exists" and "retry with backoff"; "Provider" with "in the status code" and "depends on the code"; "Model" with "only when you validate content" and "validate, repair, or reject". A vertical bracket spans the first three rows labelled "your code is wrong". A second bracket spans the last four rows labelled "the world is unreliable". A small note beside the "Model" row reads "everything else reports success".
Required elements: Seven labelled rows, three columns with the stated headers and cell text, two vertical brackets with their labels, the note beside the model row.
Style: Clean educational illustration, professional, uncluttered, high contrast, flat vector, clear table gridlines and generous row spacing.
Layout: Table reading top to bottom, brackets on the left edge spanning their row groups, note to the right of the final row.
Text labels: "Failure type", "When you find out", "Correct response", "Syntax", "Runtime", "Logical", "Data", "Network", "Provider", "Model", "before running", "during execution, deterministic", "never, unless tested", "at the input boundary", "before a response exists", "in the status code", "only when you validate content", "fix the code", "tests and evaluation", "reject with a reason", "retry with backoff", "depends on the code", "validate, repair, or reject", "your code is wrong", "the world is unreliable", "everything else reports success".
Aspect ratio: 4:3
Accessibility: Convey the two groupings through labelled brackets and row order rather than colour alone.
Avoid: Decorative icons, screenshots, tiny text, logos, watermarks, clutter.
Alt text: Reference chart of seven failure types showing when each is detected and the correct response, grouped into failures caused by wrong code and failures caused by an unreliable world.
END IMAGE PROMPT]

Why the taxonomy is practical rather than academic. Every reliability decision in this module reduces to classifying a failure correctly. Retry the network error and the 503. Do not retry the 400, because the request will be malformed on every attempt. Reject the data error with a reason. Fix the runtime error in code. Validate for the model error. Get the classification wrong and you either hammer a service with requests that cannot succeed, or you give up on failures that would have succeeded on the next attempt.