Google Flu Trends Gets It Wrong Three Years Running

wabrandsma writes with this story from NewScientist: "Google may be a master at data wrangling, but one of its products has been making bogus data-driven predictions. A study of Google's much-hyped flu tracker has consistently overestimated flu cases in the US for years. It's a failure that highlights the danger of relying on big data technologies.

Evan Selinger, a technology ethicist at Rochester Institute of Technology in New York, says Google Flu's failures hint at a larger problem with the algorithmic approach taken by technology companies to deliver services we all want to use. The problem is with the assumption that either the data that is gathered about us, or the algorithms used to process it, are neutral. Google Flu Trends has been discussed at slashdot before: When Google Got Flu Wrong."

The problem is the question, not the answer

[lucm]

With big data, when you actively look for patterns you always find them; this is how hedge funds have been operating for years. The purpose of the technology is not to make predictions, but rather to confirm existing trends and possibly identify new ones.

Proper way to utilize big data in this case would be:
1) to assist the CDC in confirming or refuting trends observed in the field
2) to offer additional correlations (such as: are people living closer to highways more sensitive fo specific strains of flu)
3) to provide long-term indicators facilitating the assessment of medication and other flu containment factors

Big data is not a magic eight ball but it's not a piece of shit either.