Lesson 1: Why attention needs positions, and the absolute fix

Module 4 · Lesson 1 · About 50 minutes · Needs: Module 3, Lessons 1 and 2 (ModelConfig, Block); Module 2, Lesson 4 (the causal mask)

The situation

Sol 408 at Aurora Station. Maya Lindqvist reads line 3 of Ana Dubois's log for sol 286: "Ana recorded 715 litres before the work and 886 litres after it." Swap the numbers and the fact reverses, with exactly the same characters. "Does the model know which reading came first?" Kofi Mensah adds: every maintenance log is longer than the 128 characters base reads. "What happens past the last position it trained on?"

This lesson measures both, and starts tinyformer/positions.py with the absolute methods and their ModelConfig switch.