Incident: Hidden from people, still served to the model
Module 5 · Incident file · About 30 minutes
What happened
In August 2024 researchers at Lasso, an Israeli cybersecurity company, saw a LinkedIn post claiming that OpenAI was training on, and exposing, data from private GitHub repositories. The repository in question gave a 404 error on GitHub, because it had been made private, yet its pages were still indexed by Bing. Ophir Dror and Bar Lanyado then found that one of Lasso's own repositories, public by mistake for a short time and since made private, could still be read through Microsoft Copilot, which uses Bing. Copilot "returned actual data from the time when the repository was public". They called it "Zombie Data": data the owner thinks is private or deleted, "but is still accessible" (Lasso, 2025).
Lasso then listed repositories that had been public at some point in 2024 and were later deleted or made private. Their write-up reports 20,580 such repositories extracted through Bing's cache, 16,290 organisations affected, more than 100 internal Python and Node.js packages open to dependency confusion, and more than 300 private tokens, keys and secrets for services such as GitHub, Hugging Face, GCP and OpenAI. Lasso contacted the affected organisations and advised them to rotate or revoke the exposed keys. Amazon later told TechCrunch it was not affected (TechCrunch, 26 February 2025).
Lasso reported the issue to Microsoft in November 2024, according to TechCrunch. Microsoft classified it as low severity and called the caching behaviour "acceptable". Even so, within about two weeks Bing's cached-link feature was removed and access to the cache domain, cc.bingj.com, was disabled for all users. When Lasso looked again, "Copilot still had access to the cached data that was no longer available to human users". On 14 January 2025, after the fix, they retrieved through Copilot the content of a tool whose repository had been removed from GitHub after Microsoft took legal action against its authors. Their verdict: "while public access was blocked, the underlying data had not been fully removed." The write-up was published on 27 February 2025. TechCrunch reported that Microsoft did not respond to its questions.