Lab: The Kestrel repository and a first baseline
32 min read·
Module 1 · Lab · About 50 minutes · Needs: lessons 1 to 3 of this module
What you are building
Adaeze's question for the whole course is "Can you reproduce last month's model exactly?" That starts today, with one repository that keeps everything. The lessons wrote their files into Colab's own folder, which empties when the session ends. This lab puts them in a repository called kestrel in your Google Drive, and every later lab adds to it.
By the end the repository holds the shared data as kestrel/data.py, the three maths files from the lessons, Kestrel's first model (logistic regression written from scratch, checked against scikit-learn), and tests that fail if the data ever changes. It also has the files every repository needs: requirements.txt, .gitignore, a README.md and the decision-record template in docs/. No cell calls a hosted model, so you need no key.
Open a new Colab notebook, call it module-01-lab.ipynb, and run each cell in order.
Step 1: Open the repository in your Drive
The files must outlive the session, so the first cell connects your Google Drive and moves the notebook into the repository folder. It is safe to run twice.
python
import os
import sys
if os.path.basename(os.getcwd()) != "kestrel": # not in the repository yet
if "google.colab" in sys.modules: # in Colab: the repository lives in your Drive
from google.colab import drive
drive.mount("/content/drive")
os.chdir("/content/drive/MyDrive")
os.makedirs("kestrel", exist_ok=True)
os.chdir("kestrel")
print("working in", os.path.basename(os.getcwd()))Code walkthrough
- If the current folder is already called
kestrel, nothing happens, so a second run never nests folders. "google.colab" in sys.modulesisTrueonly in Colab. Theredrive.mountasks for access and your Drive appears as/content/drive/MyDrive.os.makedirs(..., exist_ok=True)makes the folder if it is missing andos.chdirmoves into it. Every path from here on (kestrel/...,tests/...,docs/...) is inside the repository. On your own computer the folder is made where the notebook started.
You should see
text
working in kestrelIn Colab a line Mounted at /content/drive comes first. The pop-up needs your Google account, so the course's test run took the "own computer" path.
Step 2: Add %%ensure and the skeleton
A plain %%writefile replaces a file on every rerun, and it will not make a missing folder. So the course defines %%ensure, which writes a file only when it is missing, making folders as it goes; with -a it adds a part, but only to a file this session has just written. Every later lab pastes this cell first.
python
%pip install -q pytest==9.1.1 scipy==1.18.1 scikit-learn==1.9.1python
from pathlib import Path
from IPython.core.magic import register_cell_magic
WROTE = set() # files this session's %%ensure wrote
@register_cell_magic
def ensure(line, cell):
"""%%ensure path: save the cell unless the file exists. %%ensure -a path: add to a file this session wrote."""
*flags, name = line.split()
path, text = Path(name), cell.rstrip("\n") + "\n"
if flags == ["-a"]:
if name in WROTE:
with path.open("a", encoding="utf-8") as f:
f.write("\n\n" + text)
print("added to", path)
return
if path.exists():
WROTE.discard(name) # a kept file gets no added parts either
print("kept", path)
return
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(text, encoding="utf-8")
WROTE.add(name)
print("wrote", path)Code walkthrough
%pip installpins the test runner, SciPy (the lessons' version) and scikit-learn, which checks the baseline in Step 5.@register_cell_magicturnsensureinto the command%%ensure:lineis the rest of the first line,cellthe rest of the cell.*flags, name = line.split()keeps the last word as the file name and any-ainflags.cell.rstrip("\n") + "\n"ends every part with one newline, so a file is byte-identical in Colab and on a laptop.- Without
-a: an existing file printskept; a missing one is written, folders included, and remembered inWROTE. - With
-a: the part is added after a blank line only if this session wrote the file; after akeptit is skipped. - To replace a file on purpose, delete it and run its cells again. The cell itself prints nothing: it only defines the command.
python
%%ensure kestrel/__init__.py
"""Kestrel's late-delivery risk model: the package every lesson and lab adds to (course: AI / ML Engineer)."""Code walkthrough: __init__.py makes kestrel a regular package. The lessons imported it without one (Python treats such a folder as a namespace package); the repository keeps this one-line docstring, so import kestrel always finds this folder and no other.
python
%%ensure requirements.txt
numpy==2.5.4
pandas==3.1.0
scipy==1.18.1
scikit-learn==1.9.1
lightgbm==4.7.0
xgboost==3.4.1
statsmodels==0.15.0
imbalanced-learn==0.14.2
umap-learn==0.5.12
torch==2.14.1
torchvision==0.29.1
lightning==2.6.6
transformers==5.20.2
huggingface-hub==2.3.0
datasets==5.1.0
peft==0.21.2
accelerate==1.15.0
mlflow==3.17.2
optuna==5.0.0
pandera==0.34.1
prefect==3.9.0
dvc==3.67.1
duckdb==2.0.0
pyarrow==25.0.1
fastavro==1.13.1
protobuf==7.36.2
onnx==1.23.2
onnxruntime==1.31.0
skl2onnx==1.20.0
onnxmltools==1.16.0
fastapi==0.150.0
httpx==0.28.1
uvicorn==0.54.0
pydantic==2.14.1
redis==8.1.0
fakeredis==2.40.0
evidently==0.7.23
shap==0.54.1
fairlearn==0.15.0
pytest==9.1.1
hypothesis==6.170.9
line-profiler==5.1.0
langchain==1.4.7
langchain-core==1.7.0
langchain-groq==1.1.3
langchain-google-genai==4.4.2
langchain-ollama==1.1.0
python-dotenv==1.2.4Code walkthrough: the course's pins from course-guide/07, all 48 lines, written once for the whole course. Colab installs only what each lab needs; on a laptop, pip install -r requirements.txt installs everything.
python
%%ensure .gitignore
.env
.venv/
__pycache__/
.pytest_cache/
.ipynb_checkpoints/
*.egg-info/
build/
dist/Code walkthrough: .gitignore names what Git must never store: .env (your keys, from Module 12), the virtual environment, Python's caches, and the build folders that pip makes from Module 2 on.
python
%%ensure README.md
# Kestrel: late-delivery risk
A calibrated probability that a parcel misses its promised day, used to warn customers at 06:00 and to send
spare vans (course: AI / ML Engineer). All data is simulated and generated by `kestrel/data.py`.
| Folder | What it holds |
|---|---|
| `kestrel/` | the package: data, maths, features, models |
| `tests/` | `pytest -q` runs them all |
| `docs/` | decision records (`docs/adr/`) and other design documents |
| `pipelines/`, `service/`, `evals/` | added from Modules 5, 9 and 12 |
| `reports/`, `artifacts/` | generated by code; never edited by hand |
Install with `pip install -r requirements.txt`, then run `pytest -q`.Code walkthrough: the repository's front page: what the model does, what each folder holds and the two commands that prove it works. Module 15 rewrites it as the portfolio version.
python
%%ensure docs/adr/TEMPLATE.md
# ADR-NNN: <decision in a few words>
Status: proposed | accepted | superseded by ADR-MMM
Date: <ISO date>
Deciders: <names from the cast>
## Context
<the problem, with the measured numbers that force a decision>
## Options considered
<two or three, each with quality (PR-AUC, Brier), latency, cost and what it locks in>
## Decision
<the choice, in one or two sentences>
## Consequences
<what gets easier, what gets harder, what we now have to operate or monitor>
## Revisit when
<the measurable condition that would change the decision, for example "Doncaster's PR-AUC falls 0.02 below the fleet's">Code walkthrough: course-guide/03's decision-record template, unchanged. The Module 3 lab copies it for ADR-001; every record has the measured numbers that forced the decision and a "Revisit when" condition.
You should see
text
wrote kestrel/__init__.py
wrote requirements.txt
wrote .gitignore
wrote README.md
wrote docs/adr/TEMPLATE.mdRun any of these cells again and it prints kept ...: the file was there, so it was left alone.
Step 3: Save the shared data as kestrel/data.py
Course-guide/06's blocks 1 to 8 go into kestrel/data.py, whole and unchanged: the depots, routes, weather, parcels, customer messages and the fingerprint check. From now on every lab imports the same data instead of pasting it. Block 7 is two cells, so there are nine cells; paste each as it is.
python
%%ensure kestrel/data.py
import math
import random
from datetime import date, timedelta
COURSE_DATE = date(2026, 10, 24) # versions, model names and prices checked on this day
WINDOW = (date(2026, 7, 6), date(2026, 9, 27)) # 12 weeks of parcels shipped, Monday to Sunday
DATA_AS_OF = date(2026, 9, 28) # the snapshot: an outcome not known by this day is missing
SEED = 20260928
EVENTS = { # fixed dates the data and the lessons share
"doncaster_opens": date(2026, 8, 10), # new depot DON takes routes from Leeds and Leicester
"scanner_v4": date(2026, 9, 7), # from this day weight_kg arrives in grams (the unit trap)
"highland_gales": (date(2026, 9, 15), date(2026, 9, 17)), # high winds at GLA and INV
}
BANK_HOLIDAYS = { # the two inside the window, from gov.uk/bank-holidays.json (read 24 October 2026)
"england-and-wales": date(2026, 8, 31), "scotland": date(2026, 8, 3)}
FLEET = { # the whole of Kestrel in September 2026 (fixed facts, simulated)
"depots": 9, "vans": 1_150, "parcels_per_day": 210_000, "sample_rate": 0.01,
"scores_per_parcel": 2, # scored at collection, and again at 06:00 on the promised day
}
DEPOTS = [ # id, name, nation, routes, share of parcels, effect on lateness (log-odds)
("LEI", "Leicester hub", "england-and-wales", 30, 0.16, 0.00),
("BHM", "Birmingham", "england-and-wales", 26, 0.14, 0.05),
("MAN", "Manchester", "england-and-wales", 26, 0.14, 0.10),
("LDS", "Leeds", "england-and-wales", 22, 0.12, 0.00),
("BRS", "Bristol", "england-and-wales", 20, 0.10, -0.05),
("DAR", "Dartford", "england-and-wales", 32, 0.18, 0.20),
("GLA", "Glasgow", "scotland", 18, 0.10, 0.05),
("INV", "Inverness", "scotland", 10, 0.06, 0.25),
("DON", "Doncaster", "england-and-wales", 16, 0.00, 0.20), # opens 10 August; share ramps up
]
NATION = {d[0]: d[2] for d in DEPOTS}
print(len(DEPOTS), "depots;", sum(d[3] for d in DEPOTS), "routes; window", WINDOW[0], "to", WINDOW[1])Code walkthrough: guide 06 block 1, unchanged: dates, events, the fleet and the nine depots.
python
%%ensure -a kestrel/data.py
def make_routes(seed=SEED):
"""Each depot's routes: rural or urban, island or not, typical distance and stops."""
rng = random.Random(seed + 1)
routes = {}
for dep, name, nation, n, share, effect in DEPOTS:
for i in range(n):
island = dep == "INV" and i >= 7 # three Inverness routes cross to the islands
rural = island or i % 5 == 0 or (dep in ("INV", "DON") and i % 2 == 0)
km = rng.uniform(80, 160) if island else rng.uniform(25, 60) if rural else rng.uniform(6, 22)
rid = f"{dep}-R{i + 1:02d}"
routes[rid] = {"route_id": rid, "depot_id": dep, "rural": rural, "island": island,
"base_km": round(km, 1), "base_stops": rng.randint(95, 135)}
return routes
def make_days(seed=SEED):
"""Weather per depot per day, and the ferry disruptions that are never a column."""
rng = random.Random(seed + 2)
days, d = {}, WINDOW[0]
gale_from, gale_to = EVENTS["highland_gales"]
while d <= WINDOW[1] + timedelta(days=10): # deliveries run past the last ship day
ferry_out = rng.random() < 0.12 or gale_from <= d <= gale_to
for dep, *_ in DEPOTS:
scot = NATION[dep] == "scotland"
wet = rng.random() < (0.55 if scot else 0.40)
rain = rng.expovariate(1 / (3.5 if scot else 2.0)) if wet else 0.0
wind = rng.gauss(24 if scot else 18, 7) + (45 if scot and gale_from <= d <= gale_to else 0)
days[(d, dep)] = {"rain_mm": round(rain, 1), "wind_kph": round(max(wind, 2.0), 1),
"ferry_out": ferry_out and dep == "INV"}
d += timedelta(days=1)
return days
ROUTES = make_routes()
DAYS = make_days()
print(len(ROUTES), "routes;", sum(r["island"] for r in ROUTES.values()), "island routes;",
len(DAYS), "depot-days;", sum(v["ferry_out"] for v in DAYS.values()), "ferry disruption days at INV")Code walkthrough: guide 06 block 2, unchanged: routes and daily weather.
python
%%ensure -a kestrel/data.py
DOW = [1.15, 1.05, 1.00, 1.00, 1.10, 0.55, 0.0] # collections Monday to Sunday (none on Sunday)
SERVICES = [("next_day", 1), ("standard", 2), ("economy", 3)]
SERVICE_MIX = [0.45, 0.40, 0.15]
def shares_on(d):
"""Share of the day's parcels per depot. Doncaster ramps up over 21 days from 10 August."""
s = {dep: share for dep, _, _, _, share, _ in DEPOTS}
if d >= EVENTS["doncaster_opens"]:
ramp = min(1.0, (d - EVENTS["doncaster_opens"]).days / 21)
s["DON"], s["LDS"], s["LEI"] = 0.08 * ramp, s["LDS"] - 0.05 * ramp, s["LEI"] - 0.03 * ramp
return s
def depot_effect(dep, d):
"""Doncaster's lateness drifts upwards as its volume grows (the drifting new depot)."""
base = {x[0]: x[5] for x in DEPOTS}[dep]
if dep == "DON":
base += 0.9 * min(1.0, (d - EVENTS["doncaster_opens"]).days / 42)
return base
def add_days(d, n):
"""n delivery days after d; Kestrel delivers Monday to Saturday."""
while n:
d += timedelta(days=1)
n -= d.weekday() != 6
return d
def crosses_holiday(dep, ship, promised):
"""True when the depot's bank holiday falls between collection and the promised day."""
return ship <= BANK_HOLIDAYS[NATION[dep]] <= promised
print("Doncaster share on 1 September:", shares_on(date(2026, 9, 1))["DON"],
"| its lateness effect on 21 September:", round(depot_effect("DON", date(2026, 9, 21)), 2),
"| next-day promise for a Saturday collection:", add_days(date(2026, 9, 26), 1))Code walkthrough: guide 06 block 3, unchanged: volumes, promises and the calendar.
python
%%ensure -a kestrel/data.py
def parcel_row(rng, n, ship, route):
"""The features Kestrel knows when a parcel is collected, plus its true weight."""
dep = route["depot_id"]
service, sdays = rng.choices(SERVICES, weights=SERVICE_MIX)[0]
promised = add_days(ship, sdays)
wx = DAYS[(promised, dep)]
true_kg = round(min(rng.lognormvariate(0.6, 0.9), 30.0), 2)
return {
"parcel_id": f"KP{n:07d}", "ship_date": ship.isoformat(), "promised_date": promised.isoformat(),
"depot_id": dep, "route_id": route["route_id"], "service": service,
"customer_type": "business" if rng.random() < 0.3 else "residential",
"ship_hour": rng.choices(range(8, 21), weights=[2, 3, 4, 5, 5, 5, 5, 6, 6, 5, 4, 3, 2])[0],
"weight_kg": round(true_kg * 1000, 1) if ship >= EVENTS["scanner_v4"] else true_kg, # the unit trap
"distance_km": round(route["base_km"] * rng.uniform(0.6, 1.4), 1),
"stops_on_route": route["base_stops"] + rng.randint(-15, 25),
"address_quality": round(rng.betavariate(8, 1.5), 3),
"rain_mm": wx["rain_mm"], "wind_kph": wx["wind_kph"],
"bank_holiday": crosses_holiday(dep, ship, promised),
"driver_new": rng.random() < (0.55 if dep == "DON" else 0.08),
"rural": route["rural"], "island": route["island"],
"_true_kg": true_kg, "_ferry_out": wx["ferry_out"],
}Code walkthrough: guide 06 block 4, unchanged: one parcel's features.
python
%%ensure -a kestrel/data.py
def late_logit(p, ship):
"""Log-odds that a parcel misses its promised day. Weight counts in kilograms, whatever the column says."""
dep, kg, km, nd = p["depot_id"], p["_true_kg"], p["distance_km"], p["service"] == "next_day"
z = -4.24 + depot_effect(dep, ship) + 0.45 * p["driver_new"] + 0.3 * nd
z += 0.2 * (p["ship_hour"] >= 17) + (1.3 if nd and p["ship_hour"] >= 18 else 0.0) # the 18:00 trunk cut-off
z += 1.5 * (p["address_quality"] < 0.72) # an address a driver cannot find
z += 0.9 * (kg > 15) + (0.7 if p["rural"] and km > 45 else 0.0) # two-person parcels, long rural legs
z += (1.6 if p["rural"] else 0.5) * (1 - math.exp(-p["rain_mm"] / 3)) # rain: quick to hurt, then levels off
z += 0.08 * max(0, p["stops_on_route"] - 125) # only an overloaded route slows
z += 0.06 * max(0.0, p["wind_kph"] - 35) * (2.5 if p["island"] else 1.0) # wind matters most at sea
z += 0.8 * (ship.weekday() == 0 and dep in ("DAR", "MAN", "LEI", "LDS")) # Monday backlog at four hubs
z += p["bank_holiday"] * (0.3 + 0.025 * min(km, 60)) # a holiday hurts long routes most
return z + (2.2 if p["_ferry_out"] and p["island"] else 0.0) # ferries: a cause with no column
def add_outcome(rng, p, ship):
"""What is only known afterwards: late or not, attempts (the leak), and when the label arrives."""
late = rng.random() < 1 / (1 + math.exp(-late_logit(p, ship)))
promised = date.fromisoformat(p["promised_date"])
delivered = add_days(promised, (1 + (rng.random() < 0.3)) if late else 0)
label_ready = delivered + timedelta(days=1) # delivery scans are reconciled overnight
known = label_ready <= DATA_AS_OF
p["attempts"] = (1 + (rng.random() < (0.55 if late else 0.04)) + (late and rng.random() < 0.2)) if known else None
p["delivered_date"] = delivered.isoformat() if known else None
p["late"] = int(late) if known else None
p["label_ready_date"] = label_ready.isoformat()
for k in ("_true_kg", "_ferry_out"):
del p[k]
return pCode walkthrough: guide 06 block 5, unchanged: the outcome, the leak and the late labels.
python
%%ensure -a kestrel/data.py
def make_parcels(seed=SEED):
"""Every sampled parcel shipped in the window, in ship-date order."""
rng = random.Random(seed + 3)
by_depot = {}
for r in ROUTES.values():
by_depot.setdefault(r["depot_id"], []).append(r)
rows, n, d = [], 1, WINDOW[0]
while d <= WINDOW[1]:
per_day = round(FLEET["parcels_per_day"] * FLEET["sample_rate"] * DOW[d.weekday()])
shares = shares_on(d)
for dep, share in shares.items():
for _ in range(round(per_day * share)):
p = parcel_row(rng, n, d, rng.choice(by_depot[dep]))
rows.append(add_outcome(rng, p, d))
n += 1
d += timedelta(days=1)
return rows
PARCELS = make_parcels()
labelled = [p for p in PARCELS if p["late"] is not None]
print(len(PARCELS), "parcels;", len(labelled), "labelled;", len(PARCELS) - len(labelled),
"waiting for a label; late rate", round(sum(p["late"] for p in labelled) / len(labelled), 4))Code walkthrough: guide 06 block 6, unchanged: the 147,444 parcels.
python
%%ensure -a kestrel/data.py
SAY = { # intent: ways customers put it ({p} parcel id, {d} a day, {t} a town)
"where_is_my_parcel": ["where is {p}", "any update on {p}", "tracking for {p} hasn't moved since {d}", "is {p} still coming {d}",
"no tracking info for {p}", "has {p} left the {t} depot yet", "when will {p} arrive",
"the app just says in transit for {p}", "can you tell me where {p} is", "what time is {p} coming {d}"],
"delivery_late": ["{p} was due {d} and still nothing", "{p} is late again", "third day waiting for {p}",
"{p} missed its delivery day", "promised {d}, still not here", "why is {p} always late",
"{p} should have come {d}", "still waiting for {p}, it's overdue", "you said {d} for {p}. it is now late"],
"change_delivery": ["can {p} be left with a neighbour", "please redeliver {p} to my work in {t}", "can you hold {p} at {t}",
"change the address for {p}", "can I collect {p} from the depot", "leave {p} in the porch please",
"can {p} come {d} instead", "I'm away, please don't deliver {p} {d}", "send {p} to my mum's in {t}"],
"damaged": ["{p} arrived crushed", "the item in {p} is broken", "{p} came open and something is missing",
"{p} looks like it was dropped", "{p} was soaked through", "the contents of {p} are smashed",
"box for {p} was ripped and taped up", "got {p} {d} but the screen is cracked"],
"compliment": ["the driver for {p} was lovely", "{p} came early, great service", "thanks for getting {p} to {t} so fast",
"really helpful driver {d}", "brilliant service as always", "the driver carried {p} upstairs, thank you"],
}
VAGUE = ["what is going on with {p}", "{p}??", "still nothing {d}", "can someone call me about {p}", "not impressed with {p}",
"about {p}", "this is the second time with {p}", "re {p}"] # no clear intent: annotators have to guess
OPEN = ["", "", "", "Hi,", "Hello", "hiya", "Morning.", "Dear Kestrel,", "hey", "Good afternoon,"]
EXTRA = ["I took the day off for this.", "It's a birthday present.", "The app says out for delivery.", "nobody knocked",
"This is the second time.", "The driver left a card.", "I've emailed twice already.", "I'm out {d}.", "it's medicine"]
SIGN = ["", "", "", "Thanks", "thx", "Cheers", "Not happy.", "pls sort this", "Regards", "ta"]
SHORT = {"please": "pls", "you": "u", "tomorrow": "tmrw", "thanks": "thx", "parcel": "parcle", "delivery": "delivry",
"today": "2day", "because": "cos", "for": "4", "are": "r"} # what phone keyboards and thumbs do
NEAR = {"where_is_my_parcel": "delivery_late", "delivery_late": "where_is_my_parcel", "change_delivery": "where_is_my_parcel",
"damaged": "delivery_late", "compliment": "change_delivery"} # the intent a tired annotator slips to
TOWNS = ["Leicester", "Walsall", "Salford", "Wakefield", "Bath", "Gravesend", "Paisley", "Nairn", "Rotherham"]
DAYS_SAID = ["today", "tomorrow", "on Saturday", "on Monday", "this morning", "yesterday", "on the 14th", "last week"]Code walkthrough: guide 06 block 7 (first cell), unchanged: the customer messages' parts.
python
%%ensure -a kestrel/data.py
def scruff(rng, text):
"""Abbreviations, swapped letters and missing capitals, as people type on a phone."""
out = []
for w in text.split():
if w.lower() in SHORT and rng.random() < 0.25:
w = SHORT[w.lower()]
elif len(w) > 5 and rng.random() < 0.04:
k = rng.randrange(1, len(w) - 2)
w = w[:k] + w[k + 1] + w[k] + w[k + 2:]
out.append(w)
return " ".join(out).lower() if rng.random() < 0.3 else " ".join(out)
def make_messages(seed=SEED, n=1200):
"""Simulated customer messages. Two annotators label each; they disagree on some, as people do."""
rng, names, out = random.Random(seed + 4), list(SAY), []
late_ids, all_ids = [p["parcel_id"] for p in PARCELS if p["late"] == 1], [p["parcel_id"] for p in PARCELS]
for i in range(n):
intent = rng.choices(names, weights=[30, 25, 20, 15, 10])[0]
kind = rng.choices(["clear", "mixed", "vague"], weights=[74, 15, 11])[0]
other = NEAR[intent] if rng.random() < 0.6 else rng.choice([x for x in names if x != intent])
pid = rng.choice(late_ids if intent == "delivery_late" else all_ids)
core = [rng.choice(VAGUE)] if kind == "vague" else [rng.choice(SAY[intent])] + [rng.choice(SAY[other])] * (kind == "mixed")
rng.shuffle(core)
parts = [rng.choice(OPEN), *(c + rng.choice([".", "", "!", "?"]) for c in core)]
parts += rng.sample(EXTRA, rng.choice([0, 0, 1, 1, 2, 3])) + [rng.choice(SIGN)]
said = pid if rng.random() < 0.75 else rng.choice(["my parcel", "my order", "it"])
text = scruff(rng, " ".join(x for x in parts if x).format(p=said, d=rng.choice(DAYS_SAID), t=rng.choice(TOWNS)))
if rng.random() < 0.06: # personal data a later lesson must redact (reserved ranges only)
text += f" Call me on 07700 900{rng.randint(100, 999)} or email c{i}@customer.example"
a, b = (rng.choice([intent, other]) if kind != "clear" else NEAR[intent] if rng.random() < slip else intent
for slip in (0.05, 0.12)) # annotator a slips on 5% of clear messages, b on 12%
out.append({"message_id": f"M{i + 1:04d}", "parcel_id": pid, "text": text, "label_a": a, "label_b": b})
return out
MESSAGES = make_messages()
print(len(MESSAGES), "messages;", sum(m["label_a"] != m["label_b"] for m in MESSAGES), "annotator disagreements;",
sum("07700" in m["text"] for m in MESSAGES), "contain a phone number")Code walkthrough: guide 06 block 7 (second cell), unchanged: the 1,200 customer messages.
python
%%ensure -a kestrel/data.py
import hashlib
import json
def fingerprint(*datasets):
"""The first 12 hex digits of a SHA-256 hash of the data: equal fingerprints mean identical data."""
blob = json.dumps(datasets, ensure_ascii=False, sort_keys=True, default=str).encode("utf-8")
return hashlib.sha256(blob).hexdigest()[:12]
CHECKS = { # label: the data behind it (only the blocks you pasted are checked)
"routes": lambda: [ROUTES],
"days": lambda: [[(d.isoformat(), dep, v) for (d, dep), v in DAYS.items()]],
"parcels": lambda: [PARCELS],
"messages": lambda: [MESSAGES],
"all": lambda: [DEPOTS, FLEET, ROUTES, PARCELS, MESSAGES],
}
for label, data in CHECKS.items():
try:
print(f"{label:9}", fingerprint(*data()))
except NameError: # a block this check needs was not pasted
passCode walkthrough: guide 06 block 8, unchanged: the fingerprints.
You should see
text
wrote kestrel/data.py
added to kestrel/data.py
added to kestrel/data.py
added to kestrel/data.py
added to kestrel/data.py
added to kestrel/data.py
added to kestrel/data.py
added to kestrel/data.py
added to kestrel/data.pyImport the data from the package. The import runs data.py once, which builds everything and prints each block's summary line and the five fingerprints.
python
from kestrel.data import *Code walkthrough: * brings every name in data.py into the notebook: PARCELS, MESSAGES, ROUTES, DAYS, EVENTS, fingerprint and the rest.
You should see
text
9 depots; 200 routes; window 2026-07-06 to 2026-09-27
200 routes; 3 island routes; 846 depot-days; 14 ferry disruption days at INV
Doncaster share on 1 September: 0.08 | its lateness effect on 21 September: 1.1 | next-day promise for a Saturday collection: 2026-09-28
147444 parcels; 144450 labelled; 2994 waiting for a label; late rate 0.0866
1200 messages; 301 annotator disagreements; 72 contain a phone number
routes 3167417f391b
days 80a1703dc861
parcels 40ba9d4eaca4
messages ba9d7ac7fe4f
all f0423440d615Every fingerprint matches course-guide/06, ending all f0423440d615. If one differs, a cell was changed while pasting: delete kestrel/data.py and paste the nine cells again. Then make the DataFrame with block 9, unchanged; it stays in the notebook, not in the package.
python
import pandas as pd
df = pd.DataFrame(PARCELS)
for c in ("ship_date", "promised_date", "delivered_date", "label_ready_date"):
df[c] = pd.to_datetime(df[c])
print(df.shape[0], "rows,", df.shape[1], "columns:", ", ".join(df.columns))Code walkthrough: guide 06 block 9, unchanged: one row per parcel, the four date columns as real dates.
You should see
text
147444 rows, 22 columns: parcel_id, ship_date, promised_date, depot_id, route_id, service, customer_type, ship_hour, weight_kg, distance_km, stops_on_route, address_quality, rain_mm, wind_kph, bank_holiday, driver_new, rural, island, attempts, delivered_date, late, label_ready_dateStep 4: Keep the lessons' maths
The three lessons wrote one file each, in two or three parts, and no lesson wrote another's file. So the repository keeps each exactly as its lesson wrote it, with %%ensure in place of %%writefile.
python
%%ensure kestrel/maths/linalg.py
"""Linear algebra for Kestrel, in NumPy: parcels as vectors, the table as a matrix, and what we do with them."""
import numpy as np
NUMERIC = ["ship_hour", "weight_kg", "distance_km", "stops_on_route",
"address_quality", "rain_mm", "wind_kph", "driver_new"] # 8 numbers known at collection
def as_matrix(df, cols=NUMERIC):
"""The parcel table as a float matrix X: one row per parcel, one column per feature."""
return df[cols].astype(float).to_numpy()
def score(X, w, b=0.0):
"""One number per parcel: each row of X dotted with the weights w, plus the intercept b."""
assert X.shape[-1] == w.shape[0], f"X has {X.shape[-1]} columns but w has {w.shape[0]} weights"
return X @ w + b
def fit_least_squares(X, y):
"""Weights that make X @ w + b as close as possible to y (the sum of squared gaps is smallest)."""
A = np.column_stack([np.ones(len(X)), X]) # a column of ones carries the intercept
coef, *_ = np.linalg.lstsq(A, y, rcond=None)
return coef[1:], coef[0]
def unit(v):
return v / np.linalg.norm(v)
def cosine(a, b):
"""The dot product of two directions: 1 same way, 0 at right angles, -1 opposite."""
return float(unit(a) @ unit(b))
def project(a, onto):
"""The shadow of a on the line through `onto`: (a . u) u, with u the unit vector of `onto`."""
u = unit(onto)
return (a @ u) * uCode walkthrough: Module 1 lesson 1's linear algebra, part 1 of 3, unchanged: vectors as a matrix, score, least squares, cosine and project.
python
%%ensure -a kestrel/maths/linalg.py
def standardise(X, mean=None, sd=None):
"""Each column moved to mean 0 and scaled to standard deviation 1 (mean and sd from training rows)."""
mean = X.mean(axis=0) if mean is None else mean
sd = X.std(axis=0) if sd is None else sd
return (X - mean) / sd, mean, sd
def power_iteration(C, steps=200, seed=0):
"""The top eigenvector of a symmetric matrix C: multiply by C, rescale, repeat."""
v = np.random.default_rng(seed).normal(size=C.shape[0])
for _ in range(steps):
v = unit(C @ v)
return float(v @ C @ v), v # the eigenvalue, then the eigenvector
def pca_svd(Z, k):
"""PCA by SVD of the centred matrix Z: (directions, share of variance per direction, coordinates)."""
U, s, Vt = np.linalg.svd(Z, full_matrices=False) # Z = U diag(s) Vt
share = s**2 / (s**2).sum()
return Vt[:k], share, Z @ Vt[:k].T
def low_rank(M, k):
"""The best rank-k copy of M: keep the k largest singular values, drop the rest."""
U, s, Vt = np.linalg.svd(M, full_matrices=False)
return U[:, :k] * s[:k] @ Vt[:k]Code walkthrough: Module 1 lesson 1's linear algebra, part 2 of 3, unchanged: standardise, power_iteration, pca_svd and low_rank.
python
%%ensure -a kestrel/maths/linalg.py
def norms(v):
"""Three ways to measure a vector's size: L1 (sum of sizes), L2 (straight line), L-infinity (largest)."""
return {"l1": float(np.abs(v).sum()), "l2": float(np.sqrt(v @ v)), "linf": float(np.abs(v).max())}
def pairwise_distances(P):
"""Euclidean distance between every pair of rows of P, by broadcasting (n, 1, d) against (1, n, d)."""
diff = P[:, None, :] - P[None, :, :]
return np.sqrt((diff**2).sum(axis=-1))
def scale_by_depot(X, depot_ids, depots, means, sds):
"""Standardise each parcel with its own depot's mean and sd: (n, d) minus (n, d) rows picked by depot."""
idx = np.searchsorted(depots, depot_ids) # each parcel's row in the (9, d) tables
return (X - means[idx]) / sds[idx]Code walkthrough: Module 1 lesson 1's linear algebra, part 3 of 3, unchanged: norms, pairwise_distances and scale_by_depot.
python
%%ensure kestrel/maths/optimise.py
"""Calculus and optimisation for Kestrel, in NumPy: losses, gradients, and the ways to walk downhill."""
import numpy as np
def sigmoid(z):
"""Squash any number into a probability between 0 and 1."""
return 1 / (1 + np.exp(-z))
def log_loss(w, A, y):
"""Average log loss of the probabilities sigmoid(A @ w) against the 0/1 labels y."""
p = np.clip(sigmoid(A @ w), 1e-15, 1 - 1e-15)
return float(-np.mean(y * np.log(p) + (1 - y) * np.log(1 - p)))
def log_loss_grad(w, A, y):
"""The gradient, by the chain rule: (p - y) for each parcel, times its features, averaged."""
return A.T @ (sigmoid(A @ w) - y) / len(y)
def numeric_grad(f, w, h=1e-6):
"""Central finite differences: nudge one weight up and down by h, see how f moves."""
g = np.zeros_like(w)
for i in range(len(w)):
e = np.zeros_like(w)
e[i] = h
g[i] = (f(w + e) - f(w - e)) / (2 * h)
return gCode walkthrough: Module 1 lesson 2's losses, gradients and descent, part 1 of 2, unchanged: sigmoid, log_loss, its gradient and numeric_grad.
python
%%ensure -a kestrel/maths/optimise.py
def gradient_descent(grad, w0, lr, steps, beta=0.0):
"""Step against the gradient `steps` times; beta > 0 adds momentum (a running sum of past steps)."""
w, v, path = np.array(w0, dtype=float), np.zeros(len(w0)), []
for _ in range(steps):
v = beta * v + grad(w)
w = w - lr * v
path.append(w.copy())
return w, np.array(path)
def minibatch_sgd(A, y, lr, batch, epochs=1, seed=0):
"""Gradient steps on random batches of `batch` parcels; one epoch sees every parcel once."""
rng, w = np.random.default_rng(seed), np.zeros(A.shape[1])
for _ in range(epochs):
order = rng.permutation(len(y))
for start in range(0, len(y), batch):
rows = order[start:start + batch]
w = w - lr * log_loss_grad(w, A[rows], y[rows])
return w
def newton(A, y, steps):
"""Newton's method: divide the gradient by the curvature (the Hessian) at every step."""
w, losses = np.zeros(A.shape[1]), []
for _ in range(steps):
p = sigmoid(A @ w)
H = (A.T * (p * (1 - p))) @ A / len(y)
w = w - np.linalg.solve(H, log_loss_grad(w, A, y))
losses.append(log_loss(w, A, y))
return w, lossesCode walkthrough: Module 1 lesson 2's losses, gradients and descent, part 2 of 2, unchanged: gradient_descent (with momentum), minibatch_sgd and newton.
python
%%ensure kestrel/maths/stats.py
"""Probability and statistics for Kestrel: rates, tests, intervals and information, in NumPy and SciPy."""
import numpy as np
from scipy import stats
def bernoulli_loglik(p, y):
"""Log-likelihood of a late rate p for 0/1 outcomes y: how well p explains what happened."""
k, n = float(np.sum(y)), len(y)
return k * np.log(p) + (n - k) * np.log(1 - p)
def wald_interval(k, n, z=1.96):
"""The textbook interval: estimate plus or minus z standard errors. Breaks for small n or rare events."""
p = k / n
half = z * np.sqrt(p * (1 - p) / n)
return max(p - half, 0.0), min(p + half, 1.0)
def wilson_interval(k, n, z=1.96):
"""Wilson's interval for a proportion: honest even when k is 0 or n is small."""
p, z2 = k / n, z * z
centre = (p + z2 / (2 * n)) / (1 + z2 / n)
half = z / (1 + z2 / n) * np.sqrt(p * (1 - p) / n + z2 / (4 * n * n))
return max(centre - half, 0.0), min(centre + half, 1.0)
def two_proportion_z(k1, n1, k2, n2):
"""z statistic and two-sided p-value for 'these two late rates are the same'."""
pooled = (k1 + k2) / (n1 + n2)
z = (k1 / n1 - k2 / n2) / np.sqrt(pooled * (1 - pooled) * (1 / n1 + 1 / n2))
return float(z), float(2 * stats.norm.sf(abs(z)))Code walkthrough: Module 1 lesson 3's rates, tests and intervals, part 1 of 2, unchanged: the Bernoulli log-likelihood, Wald and Wilson intervals, the two-proportion z test.
python
%%ensure -a kestrel/maths/stats.py
def permutation_pvalue(y, in_group, n_perm=2000, seed=0):
"""Shuffle the group labels n_perm times: how often is the gap in late rates at least as big as observed?"""
rng = np.random.default_rng(seed)
gap = abs(y[in_group].mean() - y[~in_group].mean())
hits = sum(abs(y[g].mean() - y[~g].mean()) >= gap for g in (rng.permutation(in_group) for _ in range(n_perm)))
return (hits + 1) / (n_perm + 1)
def false_positive_risk(alpha_half, power, prior_null):
"""Kohavi, Deng and Vermeer's Bayes rule: P(no real effect | a significant result)."""
return alpha_half * prior_null / (alpha_half * prior_null + power * (1 - prior_null))
def entropy_bits(p):
"""Uncertainty of a yes/no outcome with probability p, in bits."""
return float(-(p * np.log2(p) + (1 - p) * np.log2(1 - p)))
def cross_entropy_bits(y, q):
"""Average surprise, in bits, when outcomes y arrive and you predicted probabilities q (log loss in base 2)."""
q = np.clip(q, 1e-15, 1 - 1e-15)
return float(-np.mean(y * np.log2(q) + (1 - y) * np.log2(1 - q)))
def beta_posterior(a, b, k, n):
"""Beta(a, b) prior plus k late out of n: posterior mean and 95% credible interval."""
post = stats.beta(a + k, b + n - k)
return float(post.mean()), (float(post.ppf(0.025)), float(post.ppf(0.975)))Code walkthrough: Module 1 lesson 3's rates, tests and intervals, part 2 of 2, unchanged: the permutation test, false positive risk, entropy, cross-entropy and the Beta posterior.
You should see
text
wrote kestrel/maths/linalg.py
added to kestrel/maths/linalg.py
added to kestrel/maths/linalg.py
wrote kestrel/maths/optimise.py
added to kestrel/maths/optimise.py
wrote kestrel/maths/stats.py
added to kestrel/maths/stats.pyNow prove the files import and still give the lessons' numbers.
python
from kestrel.maths.linalg import as_matrix, pca_svd, standardise
from kestrel.maths.stats import wilson_interval
before = df[df["late"].notna() & (df["ship_date"] < "2026-09-07")] # lesson 3's parcels
don = before[before["depot_id"] == "DON"]
print("Lesson 3, Doncaster:", int(don["late"].sum()), "of", len(don), "late, 95% Wilson interval",
[round(float(v), 3) for v in wilson_interval(int(don["late"].sum()), len(don))])
Z, _, _ = standardise(as_matrix(before))
_, share, _ = pca_svd(Z, 2)
print("Lesson 1, the eight scaled features:", Z.shape, "| first two principal components carry",
f"{share[:2].sum():.1%}", "of the variance")Code walkthrough
beforeis lesson 3's slice: labelled parcels shipped before the grams arrive on 7 September.wilson_interval(k, n)is lesson 3's interval for Doncaster's late rate.as_matrixpicks lesson 1's eight numeric columns,standardisescales them, andpca_svdreturns the share of variance per principal component.
You should see
text
Lesson 3, Doncaster: 366 of 2281 late, 95% Wilson interval [0.146, 0.176]
Lesson 1, the eight scaled features: (110586, 8) | first two principal components carry 27.2% of the varianceThe interval is lesson 3's [0.146, 0.176]. The two components carry little more than two of the eight columns would: the features are not low rank, so nothing can be dropped yet.
Step 5: Write Kestrel's first model
Lesson 2 trained logistic regression by gradient descent in loose notebook cells. A model the course keeps needs a name, a home and the interface every Kestrel model shares: fit(df, y), then predict_proba(df), returning one late probability per parcel. That interface lets baselines, trees, networks and the ONNX runtime replace each other later without touching the code around them.
python
%%ensure kestrel/baseline_logit.py
"""Kestrel's first model: logistic regression from scratch (Module 1), the baseline later models must beat."""
import numpy as np
from kestrel.maths.linalg import NUMERIC, as_matrix, standardise
from kestrel.maths.optimise import gradient_descent, log_loss, log_loss_grad, sigmoid
class LogitFromScratch:
"""Eight numeric features, scaled with the training mean and sd, fitted by batch gradient descent."""
def __init__(self, lr=3.0, steps=200, cols=NUMERIC):
self.lr, self.steps, self.cols = lr, steps, list(cols)
def design(self, df):
"""A column of ones (the intercept), then the features scaled with the training mean and sd."""
Z, _, _ = standardise(as_matrix(df, self.cols), self.mean_, self.sd_)
return np.column_stack([np.ones(len(Z)), Z])
def fit(self, df, y):
_, self.mean_, self.sd_ = standardise(as_matrix(df, self.cols))
A, y = self.design(df), np.asarray(y, dtype=float)
start = np.zeros(A.shape[1])
self.w_, path = gradient_descent(lambda w: log_loss_grad(w, A, y), start, self.lr, self.steps)
self.losses_ = [log_loss(w, A, y) for w in path]
return self
def predict_proba(self, df):
"""One late probability per parcel: the interface every Kestrel model shares."""
return sigmoid(self.design(df) @ self.w_)
def weights(self):
return dict(zip(["intercept"] + self.cols, self.w_.round(3).tolist()))Code walkthrough
- The imports are the lessons' own functions:
NUMERIC,as_matrixandstandardisefrom lesson 1,gradient_descent,log_loss,log_loss_gradandsigmoidfrom lesson 2. designbuilds lesson 2's matrixA: a column of ones, then the eight features scaled with the mean and standard deviation learned infit. Validation parcels are scaled with the training numbers, never their own.fitlearns the scaling, runs 200 steps at learning rate 3 (lesson 2's settings) and keeps the loss after every step inlosses_.predict_probareturns a flat array, one probability per row.weightsnames each weight for printing.
You should see
text
wrote kestrel/baseline_logit.pyTrain it on the course's training weeks and check it against scikit-learn's answer for the same model.
python
import numpy as np
from sklearn.linear_model import LogisticRegression
from kestrel.baseline_logit import LogitFromScratch
from kestrel.maths.stats import cross_entropy_bits
labelled = df[df["late"].notna()]
train = labelled[labelled["ship_date"] < "2026-08-24"] # the course's training weeks
valid = labelled[(labelled["ship_date"] >= "2026-08-24") & (labelled["ship_date"] < "2026-09-07")]
model = LogitFromScratch().fit(train, train["late"])
print(len(train), "training parcels |", len(valid), "validation parcels")
print("training loss after 1, 20, 200 steps:", [round(model.losses_[i], 4) for i in (0, 19, 199)])
print("weights:", model.weights())
sk = LogisticRegression(C=1e6, max_iter=1000).fit(model.design(train)[:, 1:], train["late"])
print("scikit-learn agrees to 1e-3:", np.allclose(np.r_[sk.intercept_, sk.coef_[0]], model.w_, atol=1e-3))
y_va, p_va = valid["late"].to_numpy(), model.predict_proba(valid)
constant = np.full(len(valid), train["late"].mean()) # the training late rate for everyone
print("validation cross-entropy, bits: model", round(cross_entropy_bits(y_va, p_va), 4),
"| one rate for every parcel", round(cross_entropy_bits(y_va, constant), 4))
print("KP0104204, probability late:", round(float(model.predict_proba(df[df["parcel_id"] == "KP0104204"])[0]), 3))Code walkthrough
trainandvalidare course-guide/03's split: labelled parcels shipped before 24 August, then 24 August to 6 September. The test weeks from 7 September stay untouched.LogisticRegression(C=1e6)sets a penalty so weak it is effectively off, so scikit-learn fits the same model on the same scaled matrix;np.allclose(..., atol=1e-3)compares all nine weights.cross_entropy_bitsis lesson 3's average surprise per parcel.constantgives every parcel the training late rate: the simplest honest forecast, which the model must beat.
You should see
text
86014 training parcels | 24572 validation parcels
training loss after 1, 20, 200 steps: [0.3408, 0.2506, 0.2504]
weights: {'intercept': -2.757, 'ship_hour': 0.257, 'weight_kg': 0.039, 'distance_km': 0.279, 'stops_on_route': 0.607, 'address_quality': -0.436, 'rain_mm': 0.206, 'wind_kph': -0.014, 'driver_new': 0.105}
scikit-learn agrees to 1e-3: True
validation cross-entropy, bits: model 0.4167 | one rate for every parcel 0.4688
KP0104204, probability late: 0.351The losses and KP0104204's 0.351 are lesson 2's numbers, now from the package. The model beats one rate for everyone on weeks it never saw, but only a little: Module 3 finds out how much better a model can do.
Step 6: Test the data
Every later number in the course rests on the data being course-guide/06's, byte for byte. A test turns that from a hope into a check that runs on every change. These four tests use only the standard library and kestrel/data.py.
python
%%ensure tests/test_data.py
"""The shared data is course-guide/06's, exactly: if a pasted block changed, these fail before any model trains."""
from kestrel.data import CHECKS, DATA_AS_OF, PARCELS, fingerprint
EXPECTED = {"routes": "3167417f391b", "days": "80a1703dc861", "parcels": "40ba9d4eaca4",
"messages": "ba9d7ac7fe4f", "all": "f0423440d615"} # course-guide/06, block 8
def test_fingerprints():
assert {label: fingerprint(*data()) for label, data in CHECKS.items()} == EXPECTED
def test_rows_labels_and_late_rate():
labelled = [p for p in PARCELS if p["late"] is not None]
assert (len(PARCELS), len(labelled)) == (147_444, 144_450)
assert round(sum(p["late"] for p in labelled) / len(labelled), 4) == 0.0866
def test_known_at_collection_come_first():
"""The 18 features come first; the four columns known only afterwards come last."""
assert list(PARCELS[0])[18:] == ["attempts", "delivered_date", "late", "label_ready_date"]
def test_no_label_after_the_snapshot():
ready = [p["label_ready_date"] for p in PARCELS if p["late"] is not None]
assert max(ready) <= DATA_AS_OF.isoformat()Code walkthrough
EXPECTEDcopies the five fingerprints printed on course-guide/06.test_fingerprintsrecomputes all five with the data's ownCHECKSand compares them in one assertion, so a failure shows every label that changed.test_rows_labels_and_late_ratepins the counts and the 0.0866 late rate every lesson quotes.test_known_at_collection_come_firstchecks the column order: the four columns known only after delivery are the last four, which Module 3's leak guard relies on.test_no_label_after_the_snapshotproves no label is dated afterDATA_AS_OF, 28 September: the data never knows the future.
Run the tests the way every later lab will.
python
!python -m pytest -qCode walkthrough: python -m pytest runs pytest with the repository folder on Python's path, so import kestrel works inside the tests; -q prints one dot per passing test.
You should see
text
wrote tests/test_data.py
.... [100%]
4 passed in 2.39sCheck your work
Run this cell. It checks the files, the data and the baseline in one go.
python
from pathlib import Path
FILES = ["kestrel/__init__.py", "requirements.txt", ".gitignore", "README.md", "docs/adr/TEMPLATE.md",
"kestrel/data.py", "kestrel/maths/linalg.py", "kestrel/maths/optimise.py", "kestrel/maths/stats.py",
"kestrel/baseline_logit.py", "tests/test_data.py"]
missing = [f for f in FILES if not Path(f).exists()]
assert not missing, f"missing: {missing}"
assert len(Path("requirements.txt").read_text().split()) == 48, "requirements.txt should hold the 48 pins"
assert fingerprint(*CHECKS["all"]()) == "f0423440d615", "the data is not course-guide/06's"
probs = model.predict_proba(valid)
assert probs.shape == (len(valid),) and 0 < probs.min() and probs.max() < 1, "one probability per parcel"
assert np.allclose(np.r_[sk.intercept_, sk.coef_[0]], model.w_, atol=1e-3), "from scratch and scikit-learn differ"
print("all checks passed:", len(FILES), "files, data f0423440d615, baseline matches scikit-learn")Code walkthrough: the cell fails with a message naming what is wrong: a missing file, a short requirements.txt, changed data, a model that returns the wrong shape, or weights that drifted from scikit-learn's.
You should see
text
all checks passed: 11 files, data f0423440d615, baseline matches scikit-learnBreak it
Change one training parcel by 10 grams, as a slip while pasting might, and ask which check notices: the model's score, or the fingerprint.
python
import pandas as pd
changed = [dict(p) for p in PARCELS] # a copy: PARCELS itself stays as it is
changed[0]["weight_kg"] = round(changed[0]["weight_kg"] + 0.01, 2) # the first parcel, 10 grams heavier
print(changed[0]["parcel_id"], "shipped", changed[0]["ship_date"], "| weight_kg", PARCELS[0]["weight_kg"], "->", changed[0]["weight_kg"])
print("parcels fingerprint:", fingerprint(PARCELS), "->", fingerprint(changed))
df2 = pd.DataFrame(changed)
for c in ("ship_date", "promised_date", "delivered_date", "label_ready_date"):
df2[c] = pd.to_datetime(df2[c])
lab2 = df2[df2["late"].notna()]
train2 = lab2[lab2["ship_date"] < "2026-08-24"]
valid2 = lab2[(lab2["ship_date"] >= "2026-08-24") & (lab2["ship_date"] < "2026-09-07")]
model2 = LogitFromScratch().fit(train2, train2["late"])
for name, m, v in (("original", model, valid), ("changed ", model2, valid2)):
print(name, "validation cross-entropy", f"{cross_entropy_bits(v['late'].to_numpy(), m.predict_proba(v)):.6f}")Code walkthrough
[dict(p) for p in PARCELS]copies every parcel, so the change never reaches the real data.changed[0]is the first parcel in the data, shipped on 6 July, so it is a training row: the retrained model sees the change.- The same pipeline runs on both tables, and both cross-entropies are printed to six decimal places.
Run this and you will see:
text
KP0000001 shipped 2026-07-06 | weight_kg 1.31 -> 1.32
parcels fingerprint: 40ba9d4eaca4 -> cc2c35dbdb32
original validation cross-entropy 0.416669
changed validation cross-entropy 0.416669The fingerprint changes completely; the score changes in the sixth decimal place, or not at all. No metric will ever tell you that the data under a model changed. That is why tests/test_data.py exists, and why Module 6 stores this fingerprint beside every training run. Module 5 meets the large version of this: on 7 September every weight became grams, and the scores stayed plausible.
What you added to Kestrel
- A repository,
kestrel, in your Drive, withkestrel/__init__.py,requirements.txt(48 pins),.gitignore,README.mdanddocs/adr/TEMPLATE.md. kestrel/data.py: course-guide/06 blocks 1 to 8, unchanged, with fingerprints that match.kestrel/maths/linalg.py,optimise.pyandstats.py, exactly as the lessons wrote them.kestrel/baseline_logit.py: logistic regression from scratch with the sharedpredict_probainterface, matching scikit-learn to 1e-3.tests/test_data.py: four tests that the data is course-guide/06's.
Going further
- Add a test that
LogitFromScratchand scikit-learn agree, so the comparison runs on every change. - Train the baseline with
minibatch_sgd(batch 256, one epoch) and compare its weights with the full-batch ones. - Put
KP0104204throughmodel.designand work out its probability by hand from the nine weights. - Initialise Git in the folder (
git init, then commit) so the next lab's changes show as a diff.