Data Mining Book Review: The Signal and the Noise

iandnoiseNate Silver is one of the most famous statistician, thanks to his blog FiveThirtyEight, among others. He correctly predicted detailed US elections in 2012. Nate recently wrote a book about prediction, very well summarized by the subtitle: why so many predictions fail – but some don’t. The book is a journey in the world of statistical prediction through topics such as earthquake, weather, politics, chess and poker among others.

The books gives both examples of prediction successes and failures. As Nate writes, in the age of Big Data: “[…] even if the amount of knowledge in the world is increasing, the gap between what we know and what we think we know may be widening“. The only critic I can make on the book is one from a European reader. The books starts with heavy chapters on US politics and baseball. We need to persist a hundred pages before reading about non-US related topics. Also, most of the text focus on domain description rather than statistical explanations (which is in fact normal for a statistician author).

The book contains excellent advices for data miners, for example about the importance of communicating about uncertainty. Nate also warns against the effect of predictions: “In many cases involving predictions about human activity, the very act of prediction can alter the way that people behave“. The often read topic about correlation/causation is of course discussed. To conclude, Nate’s book is a journey in the real-world of statistical predictions. Although the book covers complex subjects in details, it is worth reading it!

The Signal and the Noise: Why So Many Predictions Fail — but Some Don’t


Recommended Reading