Chapter Six, of  Actionable Agile Metrics Volume II, Advanced Topics in Predictability, is titled Detecting Signals on PBCs. Finding a signal in the noise is like separating the wheat from the chaff. Adding to the pile of quotes, one of the most enduring is “Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime.” A more appropriate version in the context of this chapter might be “Give a person some data and they will see a trend. Teach a person to detect a signal and they won’t be as silly” – doesn’t have the same punch, but the new version encapsulates Chapter 6.

Up to this point, we have only used the signal detection rule that if an observation(s) crosses an Upper or Lower Natural Process Limit (UNPL and LNPL) then things are getting real. Without other tools, it is easy to fall back to “trend spotting.” Vacanti uses a plot of general tobacco use among teens from 2011 to 2018 to demonstrate how charts without signal detection tools and understanding of context are easily misinterpreted. If just look at the graph that Excel draws we would jump to the conclusion that teens are flocking to cigarettes in droves. In reality, the year-to-year changes represent natural variability, not a trend. The Excel comment was not a knock on Excel, I am a big fan of Microsoft Excel. It is a comment on human nature – hence the need for rules. My first job out of college was as a statistical analyst. Armed with a 10-key printing calculator (that was before Lotus 1-2-3 changed the world), paper, a pencil, and a HUGE eraser, I churned through data to find sales trends. I did not have the luxury of creating an x-y plot in Excel and then drawing a line — all in less than a minute. Today, putting a line on a chart is so easy that everyone does it without thinking. Without rules to interpret data it is 1 – 2- 3, it’s magic, and a trend is born.

Vacanti recounts four detection rules in Chapter 6. They are: 

“Detection Rule One: A single point outside the computed natural process limits on either the X Chart or the mR chart should be interpreted as an indication of the presence of an assignable cause that has a dominant effect.”

“Detection Rule Two: Two out of three successive values beyond one of the two sigma lines (both on the same side of the average) are likely to signal a moderate process change.”

“Detection Rule Three: Four out of five successive values beyond one of the one-sigma lines (all four on the same side of the average) are likely to signal a moderate, sustained shift in the process.”

“Detection Rule Four: Eight successive values on the same side of the average are likely to signal a small, sustained shift in the process.”

Using the Labor Participation Rate: data age 25 – 34 we can see three of the four signal detection rules in action (unfortunately, the data does not illustrate Detection Rule Two). The strongest signal in the data shown is January 2022 and is below the LNPL. Context is critical when understanding any signal. In the case of this observation, the United States was still recovering from the impact of COVID on the labor markets. Both Detection Rules Three and Four provide a weak to moderate signal of a positive change to the 

Labor Participation Rate: Age 25 – 34s. If we add the context of the numerous help-wanted signs and news of a tight labor market, context supports the signal in the data. 

The same four detection rules can be applied to mR charts. One big caveat is that the mR chart is often zero-bounded. The Lower Range Limit (when calculated) is often below zero. Given that the mR is calculated by taking the absolute value of the difference between observations, you are never going to have a result below zero. Focus on the rules when applied to the Upper Range Limit.

Vacanti indicates that Dr. Wheeler suggests that you can get away with using Rules One and Four. I recommend being cognizant of and applying all four as a good rule of thumb. Finding a lot of signals to consider beats the heck out of mindlessly asking Excel to draw a trendline through the data. 

Buy a copy and get reading – Actionable Agile Metrics Volume II, Advanced Topics in Predictability.  

Week 1: Re-read Logistics and Preface – https://bit.ly/4adgxsC

Week 2: Wilt The Stilt and Definition of Variation – https://bit.ly/4aldwGN

Week 3: Variation and Predictability  – https://bit.ly/3tAVWhq 

Week 4: Process Behavior Charts Part 1 – https://bit.ly/3Huainr

Week 5: How Much Data? – https://bit.ly/47GVP24