tl;dr: This excerpt from my upcoming book, Practical Dashboards, is the fifth in an eight-part series on determining which metrics to visually flag on a dashboard (i.e., with alert dots, different-colored text, etc.) in order to draw attention to metrics that require it. In this post, I look at the “% deviation from target” method of flagging metrics on a dashboard. I explain why, despite seeming like an improvement upon single-threshold flags, and being used on many dashboards, “% deviation from target” flags can easily mislead. In a later post in this series, I’ll introduce a more useful way to flag metrics on dashboards called the “four-threshold” method.

One of the most common ways to flag dashboard metrics that require attention is the “% deviation from target” method, whereby a percentage deviation from each metric’s target value is shown beside its current value:

This seems like an effective way to draw attention to metrics on a dashboard that require it, since metrics with large % deviations from target require more attention. Right?

Well, maybe not. Consider the following situation on a typical dashboard:

“Average Payment Processing Time” is currently 3% off target, but such a small deviation for that metric basically doesn’t matter since we only start to get cocerned if it’s 15% or 20% off target.
On the same dashboard, “% of Payments Processed Successfully” also happens to be 3% off target. For that metric, though, anything more than 1% off target is an all-hands-on-deck crisis! We’re losing money every second!

So, a deviation of, say, 3% from target could mean that a metric is basically fine and requires no attention, or it could mean that the metric is in crisis and requires immediate attention. In other words, a large % deviation from target may or may not mean that a metric requires attention, and small % deviation from target may or may not mean that a metric requires attention. Hmm…

That problem alone should disqualify this method as a useful way to draw attention to metrics that require it, but there are other problems with it.

For example, since almost all metrics on a dashboard deviate at least slightly from their target on any given day/week/month, every metric gets a red or green indicator all the time so the dashboard will always be full of red and green indicators, even when everything’s fine and nothing requires attention (I call this problem “Christmas tree syndrome”). Flagging everything on a dashboard all the time is the same as flagging nothing.

This method can also get a bit confusing for users since, for some metrics on a dashboard, a positive deviation from target is desirable (e.g., Sales) but, for other metrics, negative deviations are desirable (e.g., Expenses). This can really slow down visual scanning on a dashboard and cause users to perceive “good” deviations as “bad”, or vice versa.

Is there a better way? I think so, yes…

In the next post in this series, we’ll review the last of the four common-but-ineffective flagging methods that I see on dashboards: Good/Satisfactory/Poor ranges. After that, I’ll introduce the four-threshold flags that I now recommend since this type of visual flag doesn’t have any of the drawbacks or limitations that I list for the four common-but-ineffective types. I'll then conclude the series with a post on useful statistics for setting visual flag thresholds automatically.

To be notified of future posts like this, subscribe to the Practical Reporting email list.