Galen, listicles and the missing cases

Yesterday I started a post with a reference to the 2nd century superdoc Galen and his arrogant approach to evaluating the success or otherwise of his treatments.

Today, I just wanted to pick up on two modern versions of the same error.

First, there’s the very popular internet phenomenon of the listicle (“10 habits of successful bloggers”, “5 things that happy people do” – you know the kind of thing).

Pic from daily edge  “Worst Listicles ever: Top 10 actors who died in a fire”

Now I,  like many of you, find a good listicle compelling reading. All that thinking and analysis made simple and easy. Best British Sitcom – Tick. Shortest Ever Employment – Tick. Smallest ever dog – Tick. All that footwork done for me by some researcher.

How could it be wrong to get someone else to look at examples of successful things – campaigns, products, people – and work out what these winners have in common? How could it be wrong to try to understand success by studying success? Best practice, you might say (maybe not, but let’s leave the notion for another day).

Well here’s why drawing conclusions from successful cases is misguided: there may very well be other reasons why these particular things got to be successful; the features they have in common may have bugger all to do with why each of them got successful; those features maybe correlated with success (or at least the sample of successes you have) but not causally related to it.

The only way you can draw reliable conclusions about what features are important in bringing about success is to look at all those things that didn’t get to be successful (the “missing cases” of Galen’s medical world – the ones who died).

What does this mean for the world of business: let’s take one very common and topical example from marketing and communications. If you want to know why something goes  “viral”, then you mustn’t just look at the things which do spread far and wide; you need to look at all of the cases including those that just went….phut. Everything between them and those that acheived ubiquity.

Any analysis that tries to determine what makes a success by analysing the features of success alone is at best a good guess. Only those that use all cases or a representative sample of all cases – yes, failures as much as successes – can begin to offer reliable conclusions.

Otherwise you’re stuck in Galen’s world.