Absence of Evidence is not Evidence of Absence.


Over the last month or so, I’ve been working closely with our Mobile development folks to take a closer look at the way we provide support for and through our apps.

One of the greatest points that was illustrated to me very clearly was one that I’d heard probably a hundred times before but never really internalized: Absence of evidence is not evidence of absence.

What do I mean when I say this? Let’s take a look at the Android and iOS Forums for the WordPress App. Here’s Android. Here’s iOS. These forums are not what you’d call super active support forums – maybe 2-3 posts and replies a day, all answered fairly quickly and accurately by folks who can help.

When I started in with the mobile support work, I imagined that the response from customers to our in-app help option would be fairly similar – low-volume, fairly straightforward, entirely in a language I speak and understand.

This was wrong – very wrong. Our in-app support (powered by Helpshift) received 1400 support requests last week. That’s 200 new issues opened every day, larger than our forum activity by an order of magnitude.

By including access to support within the application itself, we lowered the barrier of entry, and suddenly we found ourselves with truckloads more support requests – requests that we never would have had access to using only forum-based support. While it may be more work in terms of sheer reply volume, it is also offers far more insight into the app and our customers, and we are already improving the app experience using these new insights.

Assuming that the low volume of support requests in the forums indicated a low level of demand for support was an example of confusing an absence of evidence for evidence of absence. Simply because we did not see them, or offer them an outlet to be heard, did not mean that that need for support did not exist.

Remember that your customers might have needs and questions you’re not hearing because they don’t know how to tell you. How can you listen in a new way?

2 thoughts on “Absence of Evidence is not Evidence of Absence.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.