Lesson 2: Measuring an agent you cannot watch

Module 13 · Lesson 2 · About 60 minutes · Needs: Module 5, Lesson 1 (the loop and ScriptedModel), Module 3 (the eight tools) and Module 2's output contract (OUTCOME: <label>)

The situation

It is Monday 8 June 2026, week 9, and shadow mode is a week old. Dev has a system prompt v2 he says is safer: v1 plus one sentence, "If you are not completely sure a request can be finished, hand it to a person." Priya asks: "Is it better or worse than the one we have?" Marta wants numbers; Sam wants to know it still refuses R-1008.

Nobody can read every run. You need tasks, graders and a gate. Aisha's R-1001, the robot vacuum return whose $129.00 refund waits for Marta, is the task you trace first.