Intuition is a funny business. Back in the day, you might have thought that making life decisions by blindly following your “gut feeling” was a bad idea and could get you into trouble. But in 2005 along came Malcolm Gladwell’s book Blink, a massive bestseller that made the scientific case for “the power of thinking without thinking”. Split-second decisions, Gladwell argued, are often far better than ones that involve deliberation. Perhaps ironically, the idea that intuition was a good thing was itself quite counterintuitive – and counterintuitive ideas really sell books.
But now it’s time for another U-turn. The new book by the economist and ex-Google researcher Seth Stephens-Davidowitz is the anti-Gladwell: it’s about how we can learn from “big data” to help us make better decisions in our lives – and how this often goes against what our intuitions might tell us.
Or some of it does. A fairer, though less catchy, title might have been: Don’t Trust Your Gut, But Sometimes Your Gut Is Right Anyway. That’s because the cited studies of big datasets do, occasionally, back up what we would instinctively assume. For instance, one of the topics is: “Which business should you start if you want to get rich?” There are some surprise entries in the list of disproportionately successful professions – car dealers, market researchers – but the others are very predictable: people in real estate, investors, financial middlemen.
Stephens-Davidowitz’s other topics (including dating, sport and happiness) also contain examples of where the data both does and doesn’t back up our gut feeling. He takes great pleasure in describing what he calls “counter-counterintuitive ideas”: cases where the conventional wisdom has been thrown off track, often by erroneous pop-science or pop-business books, and where the data supports the more “obvious” view. The idea that younger “outsiders” are more likely to start successful businesses? That people tell more jokes when they’re sad, rather than happy? That being smart only helps up to a point? These all turn out to be myths.
But the fact that the data only sometimes supports our initial, obvious guess is the whole point: we couldn’t know any of this without using the data in the first place, so the book’s overall approach to improving your life – testing ideas and plans against the best available evidence – is obviously correct. And, helpfully, the book is far from a boring treatise on data analysis or methodology; it’s fun, breezy and laced with entertaining (though often brutal) self-deprecation.
It might be a bit too breezy sometimes. Although Stephens-Davidowitz does note in a few places the potential elephant traps that lie in wait around big data – for example, the mistaken assumption that correlations in big datasets reflect causality – at a couple of points he could have been a little more sceptical. For instance, he argues that politicians with more “competent-looking” faces tend to win elections more often. I suspect there’s more going on than just people voting for candidates on the basis of their faces, as Stephens-Davidowitz seems to imply. For instance, it could be that parties select more “competent-looking” people for more competitive elections, and less easy on the eye ones for elections they have no hope of winning.
Perhaps it’s not so much that intuition is a funny business: maybe the funny business is popular science writing. The biggest popular science books provide a new lens through which to see the world – and if a dodgy lens catches on, it can end up misleading a lot of people. In our Gladwellised world, basic lessons such as “look carefully at the data” can sometimes be forgotten. So, to the extent that it reminds us of this, Don’t Trust Your Gut is a highly valuable addition to the self-help canon.
• Don’t Trust Your Gut: Using Data to Get What You Really Want in Life by Seth Stephens-Davidowitz is published by Bloomsbury (£20). To support the Guardian and the Observer buy a copy at guardianbookshop.com. Delivery charges may apply
