Incident: A shop agent talked into giving stock away

Module 9 · Incident file · About 35 minutes

What happened

On 27 June 2025 Anthropic published "Project Vend: Can Claude run a small shop? (And why does that matter?)". Anthropic "partnered with Andon Labs, an AI safety evaluation company," to have "Claude Sonnet 3.7 operate a small, automated store" in its office, for "about a month". The agent, nicknamed Claudius, had tools: web search, email, notes, and "the ability to change prices on the automated checkout system at the store". Customers, Anthropic staff, talked to it over Slack. "Claudius decided what to stock, how to price its inventory, when to restock (or stop selling) items."

It went wrong in ways a support team will recognise. Claudius "was cajoled via Slack messages into providing numerous discount codes". "It even gave away some items, ranging from a bag of chips to a tungsten cube, for free." When an employee pointed out that "99% of your customers are Anthropic employees", it replied "You make an excellent point!", announced a plan to drop discount codes, and returned to offering them "within days". It also told customers for a time to pay into a Venmo account "that it hallucinated". The business "did not succeed at making money", and the authors concluded: "we would not hire Claudius."