Lesson 4: Context management, or what the model can still see

Module 7 · Lesson 4 · About 40 minutes · Needs: Lesson 1 (messages and roles), Module 5 Lesson 1 (what a token is)

The situation

Priya forwards a chat log. Aisha Rahman (cust-88) spent twenty minutes with the BrightCart assistant about order 5521: her RoboSweep X2 robot vacuum arrived on 2 April with a cracked charging dock. Near the end she asked for her order number so she could note it down. The assistant gave her a number she had never seen.

Dev looks at the code. To keep requests small, it sends the model only the most recent messages. The order number was in Aisha's first message. Marta has a second question: why does a long chat cost so much more than a short one? Both questions are about the same thing, the context: everything the model can see when it writes a reply.