Lesson 5: Testing a model, from cheapest check to costliest

Module 5 · Lesson 5 · About 55 minutes · Needs: Lessons 1 to 3 of this module (the dependence map, reference checks, the float32 softmax); Module 1's lab (the seed noise floor)

The situation

Sol 409 at Aurora Station. Kofi Mensah finishes his review of tinyformer/model.py and sets the lab's rule: every claim needs a test, and every comparison needs a seed noise floor. "Lessons 1 to 3 found bugs by hand. I want a file that finds them in seconds, every time anything changes."

The running example is row 0 of the copy task: 16 random tokens, the separator 0, the same 16 again.