CoursePython · Problem Decomposition, Data Structures, and Typed Records · part 8 of 79
Part 8 · Problem Decomposition, Data Structures, and Typed Records

Lesson 1: Logic Bootcamp

10 min read·9 Sept 2026

Why this lesson exists

Most people who struggle with programming do not struggle with syntax. They can write a loop, a condition, and a function. What stops them is the moment between reading a requirement and typing the first line, when the requirement is still a paragraph of English and the code is still nothing.

This lesson is about that moment. It is not a Python lesson. It is a method you apply on paper, before the editor is open, and it works the same whether the language is Python, SQL, or a shopping list.

The running example throughout this module is a corpus preparation job. You have been handed a folder of scraped web pages containing recipes. Some files are empty. Some are 400,000 characters of navigation boilerplate. Many are duplicates of each other. A few are binary garbage that was saved with an .html extension by mistake. Everything downstream expects clean, identified records.

That is a paragraph of English. Here is how it becomes code.

Step 1: Restate the requirement in one sentence

Before anything else, write one sentence that says what the program does. Not how. What.

The requirement above becomes:

Turn a folder of scraped files into a list of valid, deduplicated recipe documents, plus a report explaining why every rejected file was rejected.

Writing this sentence does three things. It forces you to notice what the output actually is, which here turns out to be two outputs and not one. It exposes vague words, since "clean" meant nothing until it became "valid and deduplicated". And it gives you something to check the finished program against.

If you cannot write the sentence, you do not yet understand the requirement, and that is information. It means the next step is a question to whoever gave it to you, not code.

The questions this sentence raises. Good restatements generate questions. What makes a document valid? Is a duplicate decided by exact content or near match? Should a rejected file stop the whole run? These are the ambiguities you want surfaced now, on paper, rather than discovered at line 200.

Step 2: Break it into ordered steps

Now expand the sentence into steps, in the order they must happen. Stay in English. Resist the urge to write code.

  1. Find every file in the folder
  2. Read each file's contents
  3. Reject anything that is not readable text
  4. Reject anything empty or too short to be a real recipe
  5. Compute an identifier for each remaining document
  6. Reject anything whose content we have already seen
  7. Collect the survivors
  8. Record a reason for every rejection

The value here is in the ordering. Notice that deduplication comes after the cheap validity checks, not before. Hashing every file including the 400,000-character junk ones would waste work on documents you were going to throw away regardless. Ordering decisions like that are much easier to see in a list of eight English sentences than in a half-written function.

Notice also that step 8 threads through steps 3, 4, and 6. That is a signal, and it leads directly to the next step.

Step 3: Identify state that must be tracked

State is anything the program must remember while it runs. Working it out before you write code prevents the most common structural mistake, which is discovering halfway through that you need to remember something and bolting on a variable that does not fit.

For this job:

  • The set of content hashes seen so far, so duplicates can be detected
  • The list of accepted documents
  • A count of rejections grouped by reason
  • The current file being processed, for error messages that name it

Each piece of state raises a design question immediately. The set of hashes grows with the corpus, so how large can it get, and does that matter? The list of accepted documents holds everything in memory at once, which is fine for 200 files and not fine for two million. That second question does not need answering today, but noticing it now is what stops it becoming a crisis later.

Step 4: Choose the data structure that fits the access pattern

Each piece of state has an access pattern, meaning the way you will read and write it. The access pattern chooses the structure. Lesson 2 covers the reasoning in full, but the decisions here follow from asking one question per item: what do I actually do with this?

StateWhat you do with itStructure
Hashes seen so farAsk "have I seen this?" repeatedlyset
Accepted documentsAppend, then iterate in orderlist
Rejection countsIncrement a counter per named reasondict or Counter
Current file pathHold one value at a timea plain variable

The hash set is the important one. You will ask "have I seen this?" once per document. Asking that question of a list means scanning it, which gets slower as the list grows. Asking it of a set is effectively instant no matter how large the set gets. On 200 documents nobody notices. On two million, one choice finishes and the other does not.

Step 5: Write pseudocode

Pseudocode is the bridge. It has the shape of code and the vocabulary of English, and its purpose is to let you find structural mistakes while they are still cheap to fix.

text
seen_hashes = empty set
documents = empty list
rejections = empty counter

for each path in folder:
    try to read path as text
    if reading failed:
        record rejection (path, "not readable as text")
        continue to next file

    if content is empty or shorter than minimum:
        record rejection (path, "too short")
        continue to next file

    content_hash = hash of normalized content

    if content_hash in seen_hashes:
        record rejection (path, "duplicate")
        continue to next file

    add content_hash to seen_hashes
    add Document(path, content, content_hash) to documents

return documents, rejections

Read what this already tells you before a single line of Python exists.

The structure is flat. Every rejection ends with continue, so there is no nesting and no else branch tracking. That is a deliberate pattern called a guard clause, covered in Lesson 3.

The reason string appears in three places. Free text will drift, so those reasons should be a fixed vocabulary, which Lesson 5 handles with Literal.

Document(path, content, content_hash) has already appeared. You have decided to build a record rather than pass dictionaries around, which is Lesson 4.

None of those insights required running anything. That is the point of pseudocode.

[IMAGE PROMPT M2-1
Purpose: Show the six-step decomposition method as a workflow, and make clear that implementation is late in the process rather than first.
Visual type: Vertical workflow diagram with a feedback loop.
Prompt: A clean educational vertical workflow diagram with six numbered stages arranged top to bottom, connected by downward arrows. Stage 1 reads "Restate in one sentence" with the sub-label "what, not how". Stage 2 reads "Break into ordered steps" with the sub-label "still English". Stage 3 reads "Identify state to track" with the sub-label "what must be remembered". Stage 4 reads "Choose structures" with the sub-label "by access pattern". Stage 5 reads "Write pseudocode" with the sub-label "shape of code, words of English". Stage 6 reads "Implement" with the sub-label "first actual Python". A bracket on the left spans stages 1 through 5 labelled "on paper, no editor open". A curved feedback arrow runs from a seventh box at the bottom, labelled "Attack with edge cases", back up to stage 2, labelled "found a gap, revise".
Required elements: Six numbered stages in order with sub-labels, a left-side bracket spanning the first five, a seventh edge-case box at the bottom, a curved feedback arrow returning to stage 2.
Style: Clean educational illustration, professional, uncluttered, high contrast, flat vector, evenly sized stage boxes.
Layout: Single vertical column reading top to bottom, bracket on the left, feedback arrow curving up the right side.
Text labels: "Restate in one sentence", "what, not how", "Break into ordered steps", "still English", "Identify state to track", "what must be remembered", "Choose structures", "by access pattern", "Write pseudocode", "shape of code, words of English", "Implement", "first actual Python", "Attack with edge cases", "on paper, no editor open", "found a gap, revise".
Aspect ratio: 4:3
Accessibility: Number every stage so order is explicit without relying on position or colour, and label the feedback arrow in words.
Avoid: Decorative icons, screenshots, tiny text, logos, watermarks, clutter.
Alt text: Vertical six-stage workflow showing requirement restatement, step breakdown, state identification, structure choice, and pseudocode all happening on paper before implementation, with an edge-case attack stage feeding back into the step breakdown.
END IMAGE PROMPT]

Step 6: Implement

Only now does Python appear, and the translation is nearly mechanical because every decision was already made.

python
import hashlib
from collections import Counter
from pathlib import Path

MIN_CONTENT_LENGTH = 50


def prepare_corpus(folder: Path) -> tuple[list[dict], Counter]:
    seen_hashes: set[str] = set()
    documents: list[dict] = []
    rejections: Counter = Counter()

    for path in sorted(folder.iterdir()):
        try:
            content = path.read_text(encoding="utf-8")
        except (UnicodeDecodeError, OSError):
            rejections["not_readable"] += 1
            continue

        normalized = content.strip()
        if len(normalized) < MIN_CONTENT_LENGTH:
            rejections["too_short"] += 1
            continue

        content_hash = hashlib.sha256(normalized.encode("utf-8")).hexdigest()

        if content_hash in seen_hashes:
            rejections["duplicate"] += 1
            continue

        seen_hashes.add(content_hash)
        documents.append(
            {"path": str(path), "content": normalized, "hash": content_hash}
        )

    return documents, rejections

This works, and it is deliberately not yet good. It returns dictionaries instead of records, and it counts rejections without recording which file was rejected. Lesson 4 fixes both. Showing the intermediate version matters, because real code arrives at quality in stages rather than emerging finished.

Attacking a solution with edge cases

The final step of the method, and the one most often skipped, is trying to break what you just wrote. Do this deliberately and systematically rather than by intuition.

Work through these categories every time:

Empty input. An empty folder. The function returns an empty list and an empty counter, which is correct.

One item. A folder with a single file. No special handling needed here, but loops that track "the previous item" often break on the first iteration.

All items identical. Two hundred copies of the same file. The first is accepted and 199 are rejected as duplicates, which is correct.

Extremes. A 400,000 character file. It is accepted, because there is no maximum length check. Is that intended? The requirement said nothing about it. This is a genuine gap, found on paper.

Wrong type or shape. A subdirectory inside the folder. path.read_text() raises IsADirectoryError, which is a subclass of OSError, so it is caught and counted as not_readable. That works by accident rather than design, and it is worth an explicit check.

Unicode and encoding. A file in UTF-16, or one containing an emoji, or the same word written with a combining accent in one file and a precomposed accent in another. Those last two hash differently and will not be detected as duplicates even though a human would call them identical.

Whitespace only. A file containing 200 spaces. After .strip() it is empty, so it is rejected as too short. Correct.

Boundary values. A file of exactly 49 characters and one of exactly 50. Is the boundary < or <=? The code says a 50-character document is accepted. Confirm that is what you meant.

Three real defects came out of that list, none of which required running the program: no maximum length, directories handled by luck, and Unicode normalization affecting deduplication. Finding them on paper costs minutes. Finding them in production costs considerably more.

Practice. Take this requirement and work steps 1 through 5 on paper before writing any code: "Given a list of chat messages, produce a summary of how many messages each participant sent, ignoring system messages, and flag any participant who sent more than half of all messages."

A worked version

One sentence. Count messages per human participant and flag anyone responsible for more than half of them.

Ordered steps. Filter out system messages. Count messages by sender. Total the remaining messages. Compare each sender's count against half the total. Return counts plus flags.

State. A count per sender, and a running total. The total is derivable from the counts, so it does not need separate tracking.

Structures. Counts by sender is a lookup by name, so dict or Counter. Flagged participants is a membership question, so set.

Ambiguities worth raising. Is "more than half" strictly greater? What if there are zero non-system messages, which makes the comparison a division by zero? What if two participants each sent exactly half? Answering these before coding is the entire exercise.