Lesson 1: What to measure when an agent works alone
29 min read·13 Oct 2026
Module 13 · Lesson 1 · About 50 minutes · Needs: Module 5, Lesson 1 (the loop and ScriptedModel) and Module 3 (the eight tools)The situation
It is Tuesday 2 June 2026, week 8. Shadow mode started yesterday, and limited live is set for Monday 15 June. Priya asks the obvious question: is the agent good enough? Marta wants to know what each request costs. Sam wants to know what it tried to do that it should not have.
Dev has a proposal too: a shorter prompt that tells the agent to "be quick". On Aisha's robot vacuum return, R-1001, both versions end with the right return and the right refund queued for Marta. "It worked" cannot tell them apart. This lesson builds the seven measures that can, using the ten requests in the test queue.