Once you decide to start leveraging your existing data to unlock the value present in your support unit, the first thing you have to do is start asking questions – not just any questions, but the right questions.

If you haven’t used Google Analytics or Kissmetrics or Mixpanel before, these tools are very powerful, but they can also be overwhelming. The default Google Analytics dashboard has been described as “a dump truck of data.” 

If you go into that jungle without a question, it’ll be very easy to get lost, wander around, and never bring home the treasures that are out there and waiting for you.

The good news is that you already have a ton of really fertile ground for finding and asking great questions. The tougher news is that before you can start sharing real value with others, you have to check your own assumptions and dogma first.

I recommend using your support team’s existing beliefs around your customer base to get started on your quest. Take a minute, think back on the last two or three months, and challenge yourself to identify the big untested beliefs that power your support team. Every team is different, but at Automattic, some of our big ones would be:

• Our customers want plugins for their sites.
• Our customers struggle with domains, both purchasing and in their usage.
• Our customers speak English first and everything else a distant second.
• Our customers prefer replies from the same person, even if it takes longer to get them.

Once you have a few of these beliefs, the next step is to look at that same set of beliefs, and explicitly ask yourself, is this true?

• Is it true that our customers want plugins for their sites?
• Is it true that customers struggle with domains, both purchasing and in their usage?
• Is it true that our customers speak English first and everything else a distant second?
• Is it true that our customers prefer replies from the same person, even if it takes longer to get them?

This step is important because it helps you to get in the mindset that you need to be a really great practitioner of data driven support. Whenever someone makes an assertion about your customers or about the way they use the product, your first inclination should be optimistic curiosity.

(In the past I’ve used the term “skepticism” here, but given the more recent usage of that term, I’m getting away from it. It’s become something more negative and more aggressive than its original intent – so optimistic curiosity it is!)

Optimistic curiosity means that you assume best intent, but you’re curious about the grounding of the assertion – does it come from anecdotal information? Does it come from a personal motivation? Is there data to support it? Can we see the data? And so on.

Like the human mind, our questions are best understood by way of behavior – so consider each of your beliefs, no matter how strong, and ask yourself, what measurable behavior would our customers engage in if this belief were true? Let’s go through these examples again:

• If it is true that our customers want plugins for their site, we would expect that “plugins” would be a top search term in our knowledge base. It would also be a top tag in our chat transcripts. It would also come up more frequently than other support topics in our public forums.
• If it is true that customers struggle with domains, both purchasing and using them, then we would expect to see a greater incidence of domain related questions than we see for other similarly popular products. We’d also see more traffic to domain related support documentation.
• If it is true that customers speak English first and everything else a distant second, we would expect to see sites set to English as the distant first in terms of creation rate and traffic. We’d also expect that traffic to our English language support docs would be far greater than other languages.
• If it is true that our customers prefer support responses from the same person, even if it means waiting longer for them, we’d expect to see higher feedback scores for the products or teams who “own” tickets than the products or teams who do not.

One thing that you’re going to have to get comfortable with, as a data driven support professional, is a little slop in the system.

You’re dealing with humans and human behavior here, so you’ll never be truly certain that you’re right about something – when we start to ask ourselves about behavior that indicates confirmation of a belief or hypothesis, we’re necessarily abstracting away from the actual humans we’re discussing, and in that abstraction we’re accepting a certain amount of slop in exchange for a better understanding.

That understanding comes not from knowing something about your customers, but thinking deeply about what indicators matter. Since you’ll never know, not for sure, you instead have to pick indicators, things that will point to the actual truth even if you can never measure that actual truth.

(To read more about customer research like this, I recommend Just Enough Research by the peerless Erika Hall.)

We’ve moved from untested beliefs into if questions , and developed our hypotheses (our classic if…then statements, above.) In the next Post, we’ll talk about how to actually go find the data around those behaviors. In the third and final Post, we’ll chat about how to turn that data into an argument.

• data
• fun
• Work