I recently came across this video of US Congressmember Katie Porter using data to grill the CEO of a large pharmaceutical company about high drug prices (it’s well worth watching and only about two minutes long):

I agree with Porter that drug prices are artificially inflated and it was genuinely satisfying to watch her skillfully use data to grill a powerful person who could—and should—be doing something about it. I like Porter and how she goes after powerful interests and, no question about it, she’s a great data storyteller. So great, in fact, that she managed to overwhelm my ability to think critically about what she was actually saying.

Because Porter was making her points so compellingly, I watched the video a second time to study her techniques. During that second viewing, though, I noticed that, when she mentioned “marketing and advertising costs”, she used the number for one year but, for the other types of expenses, she used six-year totals (2013-2018). Hmm. Comparing expenses for one year to expenses for six years isn’t a valid comparison. The “marketing and advertising” bubble on her whiteboard should have been about six times larger than it was.

I restarted the video and watched it again with a more critical eye, trying to set aside my affection for Porter and her cause, and my disdain for the CEO. Two other problems jumped out almost immediately:

When Porter asked the CEO about executive compensation, he replied, “About $60M a year”, to which Porter replied, “Try $334M on for size”, as if the CEO had wildly underestimated that number. If $334M is the total for 2013-2018, then the CEO’s estimate of “about $60M a year” was actually almost bang on (in fact, he actually overestimated it a bit).
It was disingenuous to ask the CEO about expense totals for an arbitrary time period (2013-2018) as if every competent executive should know those kinds of numbers off the top of their head. She should have asked for typical yearly totals, which would have been directly relevant to the point she was making and are numbers that a CEO should be expected to know. This was a transparent attempt to make the CEO look incompetent, regardless of whether he actually was or not.

These misrepresentations aren’t enough to invalidate Porter’s entire case, but they’re not exactly minor details, either. They certainly make it easier for the CEO to write off Porter’s criticism as illegitimate and so not act on it.

My point here, though, is that Porter’s data storytelling was so good that I didn’t notice any of these problems the first time I watched the video. I’m guessing that most of the 31,000 people who’ve seen it on YouTube as of this writing didn’t either, since 100% of the comments on the first few pages are positively gushy.

I feel icky critiquing Porter and I’m still pretty sure that the pharma company’s pricing is deeply problematic, but those aren’t good reasons for my critical thinking skills to step out for a coffee break. In fact, when I like the speaker and agree with what they’re saying, those are the very times when it’s most important to check that my critical thinking skills are still fully online. Great storytelling, however, makes that harder to do. We’re brought along for such a great ride that it’s easy to forget to look down and make sure that the car is still solidly on the road.

Much has been written in recent years about how powerful data storytelling can be, and that’s certainly true. As the saying goes, though, with great power comes great responsibility (this axiom predates Spider-Man by at least 150 years, I just discovered). Someone who’s great at storytelling is, almost by definition, also great at suppressing the audience’s ability to think critically about what they’re hearing.

To be clear, I’m not suggesting that data storytelling is bad, or that it’s tantamount to manipulation. Most of the time, most people use data storytelling techniques to genuinely help audiences better understand and respond to data. It’s just that, if we get carried away and start crafting the data around the story instead of the other way around, great storytelling makes that harder for audiences to notice.

By the way…

If you’re interested in attending my Practical Charts or Practical Dashboards course, here’s a list of my upcoming open-registration workshops.