Lesson 2: Linear attention, low rank, and what they cannot learn

Module 10 · Lesson 2 · About 50 minutes · Needs: Lesson 1 (what attention costs as context grows); Module 2 (attention by hand); Module 6 (the base recipe)

The situation

Sol 431 at Aurora Station. Tariq Haddad, the Comms Officer, brings a relayed assignment sheet: six crew, their places and systems, then "Question: which system does Greta Mensah look after?" It is 442 characters long. The base model reads 128 at a time, so the link software splits it, and Greta's line ends up in a different window from the question.

Tariq has read that some attention variants cost only linear time, and asks whether one would let base read the whole sheet at once. Kofi Mensah adds his condition: measure the quality cost, with seeds.