Lesson 1: Instruments for looking inside a model
22 min read·13 Oct 2026
Module 14 · Lesson 1 · About 60 minutes · Needs: Module 2, Lessons 1 and 3 (attention maps, head ablation on GPT-2); Module 3, Lesson 2 (the residual stream); shared blocks 1, 9 and 15 to 18
The situation
Sol 452 at Aurora Station. Kofi Mensah, the Reliability Engineer, asks: "What has base actually learnt?" It writes station grammar and still cannot copy a name from its own context.
Before you point instruments at base, Kofi wants proof they work, tested where the answer is known. So you build them on a small model trained on the shared induction task, whose ceiling is 100%. The running example is one held-out row of that task.