CourseAI / ML Engineer: The Complete Course · Module 13: Specialisation · part 80 of 93
Part 80 · Module 13: Specialisation

Lab: A paper result reproduced on model v2

189 min read·
Module 13 · Lab · About 45 minutes · Needs: lessons 1 and 2 of this module, and the Module 12 lab (or nothing: Step 1 rebuilds Modules 1 to 12)

What you are building

One paper, checked on v2 Reproduce, then price the fix Active flow 1 Processing step number Model v2 validation weeks 24 Aug to 6 Sep ferry log DAYS, never a column The paper's methods schema completion an expert names it k-means clusters found without labels the cheapest fix a 06:00 rule Evidence reproduced gap 20% first experiment priced docs/specialisation.md track, reproduction, experiments 1 2 3 4
Model v2's held-out weeks and the ferry log (blue, gold) go through two of the paper's methods and one fix (teal); what reproduces and what the fix is worth become the specialisation plan; circles give the order.

Adaeze's question from Lesson 2 was "what would a specialist know that we do not?" This lab answers it for the track Kestrel chose, tabular and risk, with evidence instead of a reading list. You take one published result, Oakden-Rayner and colleagues' hidden stratification (arXiv 1909.12475, 2019: a model can fail on an important subset nobody named, by "over 20%" on clinically important subsets), and check whether it holds for Kestrel's real model, v2, with two of the paper's three methods. Then you test the cheapest fix it points to, and write docs/specialisation.md: the track, the reproduction, the experiments next in line and a reading routine.

Lesson 2 showed the effect on a stand-in LightGBM. A specialist does not trust a result until it holds on the model that actually serves, so every number here is v2's. No cell calls a hosted model.

Open a new Colab notebook, call it module-13-lab.ipynb, and run each cell in order.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside

Or open everything with a plan