CourseAI Engineering: The Complete Course · Module 3: Machine Learning Basics · part 27 of 105
Part 27 · Module 3: Machine Learning Basics

Incident: The flu model that beat nothing

13 min read·13 Oct 2026
Module 3 · Incident file · About 35 minutes

What happened

2009 2010 2011 2012 2013 2014 2009 GFT misses H1N1, model updated Jun 2011 Google adds suggested searches Aug 2011 GFT starts running high Feb 2012 symptom searches get diagnoses Feb 2013 Nature: more than double the CDC Mar 2014 Science post-mortem red bar: 100 of 108 weeks too high, 21 Aug 2011 to 1 Sep 2013
What happened, in order. Google changed its search product twice (grey: outside the model), GFT began running high in August 2011 and stayed high for 100 of 108 weeks (red: failure), Nature reported it in 2013 and the post-mortem came in 2014 (blue: the analysis).

Google Flu Trends (GFT) estimated how much flu was circulating in the United States from what people searched for, and was built to track the Centers for Disease Control and Prevention (CDC), whose figures arrive weeks late. In February 2013 Nature reported that GFT "was predicting more than double the proportion of doctor visits for influenza-like illness (ILI) than the Centers for Disease Control and Prevention" (Butler, Nature, 13 February 2013; quoted in Lazer and colleagues, 2014). A year later David Lazer, Ryan Kennedy, Gary King and Alessandro Vespignani published the post-mortem in Science, "The Parable of Google Flu: Traps in Big Data Analysis" (14 March 2014). Their figure shows that from 21 August 2011 to 1 September 2013 GFT "reported overly high flu prevalence 100 out of 108 weeks", and that it "overshot the actual level in 2011-2012 by more than 50%".

The sharpest finding was about the baseline. "Even 3-week-old CDC data do a better job of projecting current flu prevalence than GFT." Mean absolute error was 0.486 for GFT against 0.311 for a simple model on lagged CDC data. The complex model had been losing to the simplest alternative for two years, and nobody had been checking.

The rest of this course is yours to keep

This course is bought on its own, once, and stays readable afterwards, including the parts added to it later.

Buy this course · ₹399See what is inside