Incident: A rule with no edges, applied everywhere

Module 2 · Incident file · About 30 minutes

What happened

On 23 February 2024 Prabhakar Raghavan, a Senior Vice President at Google, published "Gemini image generation got it wrong. We'll do better." (Google, 2024). Three weeks earlier Google had launched image generation in the Gemini app, including images of people. Raghavan wrote that "some of the images generated are inaccurate or even offensive". The Verge reported on 22 February that Gemini had been "creating diverse images of the US Founding Fathers and Nazi-era German soldiers", and that a request for "a US senator from the 1800s" returned what appeared to be Black and Native American women (Warren, The Verge, 2024).

Google paused image generation of people. According to The Verge, the pause came less than 24 hours after Google apologised for the historical images. Raghavan's post promised that the improved version would come only after "extensive testing". It returned on 28 August 2024, more than six months later, as an early-access feature built on Imagen 3, after what Google called "improved evaluation sets, red-teaming exercises and clear product principles" (Citron, Google, 2024).