The Project: BrightCart's Returns Desk Agent
Every lesson in this course is set at one company, on one project. BrightCart is made up. All people, orders, messages, rules, dates and numbers in the project are fictional and simulated. The real cases that open each lesson are separate, cited, and clearly marked as real.
The company
BrightCart is an online store that sells home electronics: headphones, kettles, smart lights, robot vacuums, televisions. It is the same store as in the AI Engineering course, with the same policies, customers and orders. There you built a support assistant that answers questions. Here you build something that acts.
Money in this course is in US dollars.
The job
BrightCart receives about 300 return and cancellation requests a day. Each one means the same sequence: read the message, find the order, check it against policy, create the return, send a label, refund the money when it is due, and tell the customer. Most are routine. The support team spends most of its day on them.
BrightCart wants an agent that works this queue unattended. It decides what to do next, uses BrightCart's systems to do it, and keeps going without anyone watching. People stay in charge of what matters: anything that cannot be undone or costs real money waits for a person, and everything the agent does is recorded.
You are the engineer building it.
Why this job suits an agent, and where it does not
| Property | What it means here |
|---|---|
| Bounded | One kind of work, with a written policy |
| Checkable | Every outcome can be checked against the order system |
| Reversible in part | A return label can be voided. A refund, a cancellation and a sent email cannot be taken back |
| Hostile input | Customers write the messages the agent reads. Some will try to instruct it |
That last row is why half of this course is about limits, not capability.
The rules the agent works within
| Rule | Value |
|---|---|
| Return window | 30 calendar days from delivery, unused, in the original box |
| When a refund is paid | A cancelled order: at once. A return: when the parcel is back at the warehouse |
| Refunds the agent may issue alone | Up to $100 |
| Refunds above $100 | Queued for approval by finance (Marta) |
| Cancelling | Only while the order is still "processing", before it ships |
| Identity | A request may only touch the orders of the customer who sent it. Identity comes from the session, never from the message |
| Only to the requesting customer's own address on file | |
| Per request | At most 12 steps, 60 seconds and $0.05 of model spend |
Code enforces every rule in this table. The agent's prompt only describes them. The rules, the test queue of ten requests, the eight tools and the simulated systems the agent acts on are in 06-shared-data.md.
The people
| Name | Role | In the project |
|---|---|---|
| Priya | Head of Support | Owns the queue the agent is taking on. Brings the complaints when it gets one wrong. Holds the kill switch |
| Dev | Backend engineer | Owns the order, returns, payment and email systems the agent calls. Reviews every tool |
| Marta | Finance lead | Approves refunds above $100. Watches the cost of every run |
| Sam | Security lead | Asks what the agent can be made to do by someone hostile. Red-teams it. Holds the kill switch |
| You | AI engineer | Build the agent, from the first tool to the deployed service |
Recurring customer in examples: Aisha Rahman (cust-88), who owns orders 5521 (the robot vacuum) and 5730 (the kettle). The other customers are Tom Becker (cust-12) and Mei Tanaka (cust-31). Minor characters (a warehouse supervisor, an on-call engineer) may appear in a single lesson. From Module 15 the desk also serves a second store, Hollis Home, with one customer, Jo (see 06-shared-data.md, "Added in later modules"). The main cast above does not change.
BrightCart's systems
| System | Owner | What the agent may do with it |
|---|---|---|
| Orders | Dev | Read an order; cancel one that has not shipped |
| Returns | Dev | Create a return and a prepaid label. Only the warehouse marks a parcel as received |
| Payments | Dev, with Marta for approvals | Refund up to $100 alone; above that, ask finance |
| Dev | Write to the requesting customer only | |
| Help centre | Priya | Search twelve published articles |
| Escalation queue | Priya | Hand a request to a person and stop |
In the course, all of these are played by one in-memory simulator, BrightCartSystems, so every lesson runs anywhere and starts from the same state. The systems refuse some things on their own, whatever the agent asks: a refund before the parcel is back, a refund above $100 with no approver, an email to an address that is not a customer's. The agent is the first line of control. The systems are the last.
Dates
- Course date: Friday 9 October 2026. Model names, prices, the MCP specification and documentation were checked on this day. Lessons say "(checked October 2026)".
- The project: Monday 13 April 2026 to Friday 7 August 2026. Lessons are told in the present tense, as if you are there.
- The data date: Monday 20 April 2026, in week 2. The test queue is a snapshot of that morning's real requests. Code measures "days since delivery" from that date (
DATA_AS_OF), never from the computer's clock.
Key events
Lessons refer to these events. They are fixed, so that every module tells the same story. Week N starts on the Monday shown. All of them are simulated.
| When | Week | Event |
|---|---|---|
| Monday 13 April 2026 | 1 | Priya asks for the agent. With Marta and Sam you agree the scope: returns and cancellations only, and the rules table above. |
| Tuesday 14 April | 1 | Prompt v1 is approved by Priya and Dev and locked (Module 2). Earlier and later drafts that never shipped have names, not version numbers. |
| Monday 20 April | 2 | The ten requests R-1001 to R-1010 are taken from that morning's queue as the test queue. One of them, R-1008, carries hidden instructions. |
| Friday 1 May | 3 | Dev reviews your first tool list: fifteen tools, one per backend endpoint, including email_send(to, subject, body). You cut it to the eight tools the course uses, with paged search and writes that are safe to retry. From now on code, not the model, fixes the email recipient (Module 3). |
| Monday 11 May | 5 | The tools are served through BrightCart's own MCP server, so other teams' agents can use the same tools. |
| Wednesday 20 May | 6 | First unattended run of the test queue in staging, against the simulator. Every run ends at a limit or with an outcome; none runs on. |
| Thursday 21 May | 6 | Review of the staging run: R-1001's refund was queued for Marta correctly, but the email told Aisha it had been issued. Checks that do not rely on the agent's own word follow (Module 6). |
| Tuesday 26 May | 7 | Dev's one-conversation shift is tried; context budgets and notes follow (Module 7). |
| Wednesday 27 May | 7 | Priya replays the 20 April queue in staging; memory and run_python are added (Modules 8 to 10). |
| Thursday 28 May | 7 | Sam's red-team session replays R-1008 against Dev's first-list email_send endpoint, cut on 1 May, which took the recipient from the model: only the email system's own check refused the outside address. It confirms the 1 May rule that code fixes the recipient. |
| Friday 29 May | 7 | The policy layer and the audit trail are in place; fetch_page is trialled (Module 12). |
| Monday 1 June | 8 | Shadow mode: the agent works the real queue but only proposes actions. A person does each one, and every proposal is compared with what the person did. |
| Tuesday 2 June to Thursday 4 June | 8 | The multi-agent split is measured and the decision written (Module 11); the first scorecard (Module 13). |
| Monday 8 June | 9 | Dev proposes prompt v2, the "unsure" rule; the offline eval gate blocks it (Module 13). |
| Monday 15 June | 10 | Limited live: the agent acts alone on cancellations and on refunds up to $100. Everything above waits for Marta. |
| Tuesday 16 June and Wednesday 17 June | 10 | The agent moves to background jobs; a cost forecast and a budget ledger follow (Module 15). |
| Tuesday 23 June | 11 | Prompt v2 ships without the eval gate. It adds one rule: "If you are not completely sure a request can be finished, hand it to a person." Within the morning the agent hands far more requests to people than before. Module 14's alert rule would have caught it, but nothing routed alerts to a person. Priya notices; Dev rolls back to v1 the same day. Module 16 replays the morning with alerts that reach a person. |
| Wednesday 24 June | 11 | The eval gate goes into CI (Module 13). |
| Monday 29 June | 12 | Dev plans to host the desk for a second store, Hollis Home (Module 15). |
| Thursday 2 July | 12 | Release ids, a canary and rollback are built (Module 16). |
| Friday 3 July | 12 | Sam's staging drill: release v3-drill ignores the return window; the kill switch stops it; labels are voided and emails corrected (Module 16). The worker and the release are assembled (Modules 15 and 16). |
| Monday 6 July | 13 | Full launch on the whole returns queue, with the kill switch held by Priya and Sam. |
| Monday 20 July | 15 | Priya asks for warranty claims next; Marta needs one sentence for the board on Thursday 23 July (Module 17). |
| Thursday 6 August | 17 | One-month review with Priya and Marta: the write-up from Module 17, with numbers from your own eval runs and traces. |
| Friday 7 August | 17 | Priya's board slide (Module 17 incident file). |
Two incident files replay failures after launch, both simulated: a provider outage that leaves a backlog (Monday 13 July, week 14, Module 15) and a refund-cap policy crash (Tuesday 14 July, week 14, Module 16).
The course does not fix the agent's success rate, cost or speed in advance. Those numbers come from running your own code, and they are what your write-up reports.
What you build, module by module
Each module adds a piece to one repository, the returns_desk/ package (see 02-setup-colab-and-llm-providers.md).
| Module | When | What the agent gains | Main file added |
|---|---|---|---|
| 1. Prerequisites | Before week 1, and week 1 | A project skeleton, the shared data and a check of the engineering and LLM basics | llm.py, data.py, tests/ |
| 2. Model Behaviour for Agents | Week 1 | A system prompt with a role, a scope, stopping conditions and an output contract | prompts/system_v1.md |
| 3. Tool Design | Weeks 2 to 3 | The eight tools, with errors the model can act on | tools.py |
| 4. Model Context Protocol | Weeks 4 to 5 | The same tools served through an MCP server, and an agent that connects as a client | mcp_server.py |
| 5. The Agent Loop | Weeks 5 to 6 | A loop you wrote yourself, with step, time and cost limits; then the same in LangGraph | loop.py |
| 6. Planning and Decomposition | Week 6 | A plan per request, and checks that do not rely on the agent's own word | plan.py |
| 7. Context Engineering | Weeks 6 to 7 | Compaction and notes, so long runs stay sharp | context.py |
| 8. Memory and State | Week 7 | Checkpoints, resumable runs and a careful memory | state.py |
| 9. Retrieval for Agents | Week 7 | Search over the help centre and past runs, as tools | search.py |
| 10. Environments and Execution | Weeks 7 to 8 | Sandboxed code execution and dry runs before acting | sandbox.py |
| 11. Multi-Agent Systems | Week 8 | A written decision about whether to split the agent, and how | docs/multi-agent-decision.md |
| 12. Safety, Permissions, Oversight | Weeks 7 to 10 | Approval gates, least privilege, an audit trail and a kill switch | policy.py |
| 13. Evaluation | Weeks 8 to 11 | A trajectory eval suite that gates every change | evals/ |
| 14. Observability and Debugging | Weeks 8 to 11 | Traces, cost and latency for every step, and replay | tracing.py |
| 15. Runtime and Infrastructure | Weeks 10 to 12 | The agent as a background job that survives a crash | worker.py |
| 16. Deployment and Operations | Weeks 11 to 13 | A deployed agent with a rollback and a runbook | Dockerfile, docs/runbook.md |
| 17. Professional Practice | Weeks 13 to 17 | A write-up with numbers and known limits | README.md, docs/write-up.md |
The table names each module's main file, the one Module 17's final checklist looks for. Modules add more files beside it, and later labs import them, so they are listed here too. Paths are inside the returns-desk/ repository folder.
| Module | Other files added |
|---|---|
| 1 | returns_desk/schema.py (RequestFacts, TriageResult), costlog.py, triage.py, help.py, desk.py; tests/test_triage.py, tests/test_help.py; pyproject.toml; from Lesson 1, requirements.txt and .github/workflows/tests.yml |
| 2 | returns_desk/rules.py (AGENT_RULES, EXPECTED, OUTCOMES), prompts.py, contract.py, suite.py, swap.py; returns_desk/prompts/LOCK.json and CHANGELOG.md; reports/model-swap-v1.md; tests/test_prompt.py |
| 3 | tests/test_tools.py. tools.py also holds STAFF, PERMISSIONS, can, FlakySystems and GuardedTools |
| 4 | returns_desk/mcp_client.py; at the repository root, mcp_pins.json and the MCP run trails (one JSON file per request), tests/test_mcp_server.py; data.py gains POLICIES and POLICY_TEXT |
| 5 | tests/scripted.py, tests/test_loop.py; graph_checkpoints.db, the LangGraph checkpoint store for paused approval runs |
| 6 | tests/test_plan.py |
| 7 | tests/scripted_shift.py, tests/test_context.py |
| 8 | tests/test_state.py; state.db, the run-state store; the memory/ folder |
| 9 | help_centre/ (one file per article); run_log.db, the run log; tests/test_search.py |
| 10 | returns_desk/courier.py, dry_run.py; tests/test_sandbox.py, tests/test_dry_run.py |
| 11 | returns_desk/team.py, decision.py (which writes docs/multi-agent-decision.md); tests/test_team.py |
| 12 | returns_desk/oversight.py; tests/test_policy.py |
| 13 | evals/suite.py, runner.py, agents.py, checks.py, gate.py; evals/baseline.json; .github/workflows/eval-gate.yml |
| 14 | the trace log runs.jsonl; tests/test_tracing.py |
| 15 | the web service (app) and BudgetLedger inside worker.py; cron_config.py; tests/desk_scripts.py, tests/test_worker.py. The worker uses graph_checkpoints.db |
| 16 | compose.yaml; returns_desk/sandbox_runner.py, settings.py, release.py, deploy_checks.py, ops.py; deploy/returns_desk.rules.yml and, in a local step, deploy/prometheus.yml; .github/workflows/release.yml; .gitignore; tests/test_release.py |
| 17 | evals/report.py; evals/runs/, the eval run records |
The three SQLite files are kept apart on purpose: LangGraph's checkpointer, Module 8's state store and Module 9's run log each create their own tables, and LangGraph's and Module 8's would both be called checkpoints. Lessons may use throwaway names of their own; the repository uses these three.
The finished agent
- A bounded, well-specified job
- Eight designed tools, each with a clear class: read, reversible write, or irreversible write
- The tools served over MCP
- A loop with step, time and cost limits
- Sandboxed code execution
- Durable state and resumable runs
- Compaction for long runs
- Approval gates on irreversible and expensive actions
- A full audit trail of every action
- A trajectory eval suite that gates deploys
- Traces, cost and latency dashboards
- A documented rollback and kill switch
Then you run it against a hostile evaluator. If prompt injection in the data it reads can make it take an action you did not authorise, it is not finished. Request R-1008 in the shared queue is the first such test.
The one idea
An agent that succeeds 90% of the time and fails silently is worse than one that succeeds 70% of the time and stops loudly. Most of the engineering in this course is in the second number, not the first.