Lesson 2: The residual stream, where every layer adds its part

Module 3 · Lesson 2 · About 45 minutes · Needs: Module 3, Lesson 1 (ModelConfig, FeedForward); Module 2, Lessons 4 and 5 (masks, GroupedQueryAttention)

The situation

Sol 406 at Aurora Station. Kofi Mensah reads shared block 17: x = x + self.attn(self.norm1(x)). "Why add? If every layer only adds, which one did the work? Show me what breaks without the plus sign."

This lesson assembles the block for tinyformer/model.py, measures what each part writes, and removes the plus sign. The running example is lines 3 and 4 of Ana Dubois's log for sol 286 (Lesson 1's log): 101 characters ending "Ana noted that pressure held steady."