I took a lot of statistics, quantitative analysis, and math courses in university. I think it was a function of going to three schools during every available semester and building up a boatload of credits before LSU gave me a diploma and made me leave the state. I still remember the day I learned partial differential equations (I finally could understand the footnotes in my economics texts). With all of that, I was not exposed to the idea of a base rate fallacy (known also as base rate bias) until several years later when I was working in the garment industry. Twice this week I have run into scenarios that are examples of base rate fallacies which suggest that many people either don’t understand the concept or are blinded by raw numbers (a shiny object phenomenon).
While both examples are important, the one that is more pertinent to this column was a study done by a former client that found that 7 projects in their organization using agile had failed (their words and definition), and 4 that did not use agile failed. When presented, several learned and well-meaning leaders in the room launched a campaign to roll back the agile changes and embrace more structured project management approaches. When the initial din died down the presenter acknowledged that the organization had completed approximately 850 projects using agile and 14 that had not used agile. The raw numbers had captured the presenter’s attention which blinded them to the base rate of agile/non-agile usage. This is the heart of the base rate fallacy. Yes, more projects using agile failed (again we are using their word), but if we compare the rates (less than 1% failure rate compared to about a 29% failure rate in the non-agile group of projects). Even if we assume the presenter did not have an agenda, the lack of sensitivity to the base rates created a scenario that begs for misinterpretation. Once a misinterpretation begins to circulate they take on a life of their own – the information establishes an anchor bias. In this case, I am told that a Slack discussion on the level of failures in agile projects was still going on several days later with many ignoring the differences in the base rates of usage.
Takeaways
- It is easy to fall prey to base rate fallacies. When presented with specific occurrence information make sure you understand the prevalence of the scenario that generated the occurrences (for example the total number of agile projects). If presented with the raw occurrences, ask out of how many.
- Everyone that creates or consumes quantitative information needs to understand and be periodically reminded of a few basic statistical and cognitive biases.
Picking the next Re-read Saturday book.
There is no time like the present to begin the process of picking the next book. The poll:
You have two votes, use them wisely. Feel free to vote again tomorrow to make your point. We will read or re-read the two books that get the most votes. The pol will remain open through Aug 31, 2021. https://poll.fm/10902769
