Untangling skill and luck [Mauboussin]


In the months following my most recent post about performance, I’ve been noodling on one key aspect of the challenge: if we take a more outcome (rather than output)-based approach to evaluating performance, how do we separate outcomes caused by luck and outcomes caused by skill? 

I started off reading Annie Duke’s “Thinking in Bets” which have been sitting in my queue and received good feedback from colleagues. I finished the book somewhat disappointed having gained some good insights on pursuing the truth and having a stronger decision-making process, but not a lot that pertained to the assessment of performance and untangling luck and skill. However, Duke did mention another book with a rather promising title by Michael Mauboussin:

The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing

Random side note, the cover art for the two books looks disturbingly similar.

Weary of committing another big chunk of time to a book on this topic, I decided to look for more lightweight mediums and was able to find this Talks at Google video for the more audio-visual inclined and this 25iq blog post for the more textual inclined. Both provide good summaries of the major themes covered in the book. There’s a lot there, including lots of interesting tangents in their own right, but I’ll try to focus on one arc that’s relevant to my own area of inquiry. 

Different domains of performance fall on a spectrum between pure luck and pure skill, but all of them have a combination of some luck and some skill.  

Source: The Success Equation

As a domain evolves, it becomes more dependent on luck than skill. That’s not because skill matters less, but because knowledge dissemination happens more quickly and cheaply, causing skill to be distributed more uniformly. Mauboussin refers to that phenomenon as “The Paradox of Skill”. 

Source: The Success Equation
The difference between Olympic Men’s Marathon 1st and 20th place times shows a similar pattern

The strategy for “how to get better” also varies depending on where the performance domain falls on the luck-skill spectrum. The closer it is to the skill edge of the spectrum, a deliberate practice strategy will yield better outcomes. The closer it is to the luck edge of the spectrum, the emphasis needs to be on a strong decision-making process. The latter helps frame Duke’s book more clearly: since poker falls closer to the luck edge of the spectrum, the heavy emphasis on the decision-making process makes a lot of sense. 

It’s worth noting, however, that this point does not seem to be corroborated with a lot of evidence, at least in the resources that I reviewed (it may be treated differently in the book). 

Mauboussin offers the following criteria for evaluating the process: 

  • Analytical — finding edge and figuring how much to bet on that edge.
  • Behavioral — understanding the common biases we all tend to fall for and weaving into the process methods to mitigate and manage those.
  • Organizational — avoiding “agency costs” (misalignment of incentives). Is the organization helping or impeding the quality of the decision?

So where does all of this leave us with regards to performance management? 

  • It supports the claim that variance in outcomes may have more to do with luck than skill. 
  • This gets compounded in more mature domains where the “Paradox of Skill” is in full effect. 
  • It supports the shift from focusing on the outcome to focusing on the process when evaluating performance. 

Did it solve our problem? No. Did it get us closer to a solution? Yes. Baby steps…

Untangling skill and luck [Mauboussin]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s