Tag: supconf

Full SupConf Video for Use the Data You Have

Hey folks! I’m really happy to share with you the full video of my recent talk at SupConf 2016, where I gave a talk on leveraging your existing data to build value from your support organization – here it is!

I also created a supplementary Page with a three-parter on how to execute the ideas I present in the talk – you can find that at https://s12k.com/supconf/ .

SupConf East is coming soon – as a speaker and an attendee, I cannot recommend it highly enough! You can get updates on the next SupConf event by signing up for the mailing list here.

Use the Data You Have: Presenting and Persuading

You’ve arrived at the third and final Post in this series (Use the Data You Have, following my presentation at SupConf 2016) – if you’re just starting now you may want to check out the previous few Posts, covering the importance of data in being a successful support professional, asking the right questions, and one way to approach answering those questions.

We’re arriving now at the crux of my talk – how to use the answers you’ve found to persuade others within your organization to add value for you, your customers, and your organization’s bottom line.

I could talk a lot about the importance of data visualization in persuasion and digital charisma – and likely there are many Posts in the future on that topic – but for now let’s focus more on the bigger approach, and less on whether to use a histogram or a pie chart.

(Not to belabor the point, but use a histogram.)

Many folks rush to the more sexy idea of visualization before they ask the bigger questions, and building the right foundations. It doesn’t matter how pretty your animated d3.js donut charts are if the underlying data is not something your audience cares about.

At this point in this series, you’ve considered your biggest beliefs as a support professional in your organization, you’ve converted those beliefs into hypotheses, and you’ve confirmed or denied those hypotheses using your company’s existing data, be it through Google Analytics or Mixpanel or whatever.

Now, as a data driven support professional, you’ve arrived at the hard part, at the part that I can only guide you through in a general way, because I lack the tribal or communal understandings of your workplace.

You need to find a way to explain this data to the folks who can enable change in a way that is motivating to them. This means setting your own ego and possibly your own perspective aside in the pursuit of being persuasive – folks in your organization are going to have problems and motivations that may be alien to you, but in presenting an argument, sharing a victory for both of you is far more important than being 100% true to your own perspective.

Sometimes this means going back to the drawing board – sometimes you need to do some more digging to find information that will speak to different parties. This is OK. Better to do more research than not enough – at least in this situation.

(There are times when enough is enough, for now, I’ll trust you all to know when you’re in an unproductive research rabbit hole.)

Ask yourself: what is most important to this decision maker today?

Then, figure out how to show them that the issue you’re championing can have a direct impact on what matters to them.

Are you in a high-growth startup, where moving the Monthly Active Users needle is the very most important thing? If so, you need to see how your issue can impact that needle; what does Active mean? Do Active users tend to experience this problem? If so, how can you reduce it? If not, is this issue the blocker for more Active users?

Are you in a mature company, struggling with turbulent retention rates? Show how this issue is related to or not related to customers retention.

The name of this short series is Use the Data You Have, and the importance of this cannot be understated: if you need to run a test or an experiment to verify that something needs to be solved or addressed, then you’re approaching it the wrong way. Big problems, problems that deeply need solving, are problems because they manifest in some way.

Go into your archives. Dig into your analytics suite. Find that manifestation and use it to enact positive change. Good support teams answer customers. Great support teams solve problems, and in so doing, build value for the customer and for the company.



Use the Data You Have: Answer Your Questions

As discussed in the Previous Post in this series (Ask the Right Questions), before you set foot in your Analytics suite, you need to have some idea of the questions that you want to answer.

Eventually, when you’re a superstar with your analytics toolbox, you’ll be able to do some exploratory analysis  – jumping in without a hypothesis or a ready understanding of what you’re looking for. For your first steps as a data driven support professional, I’d recommend having your question (or questions) ready to go.

For the purposes of this series, we’ll do a (very) brief overview of navigating through Google Analytics, and a tiny bit on Mixpanel. It’s important that you become a confident and competent practitioner of your particular toolset. If it’s Google Analytics, get certified.

(Here’s mine!)

Let’s consider the hypothesis from our last Post;

If it is true that our customers want plugins for their site, we would expect that “plugins” would be a top search term in our knowledge base. It would also be a top tag in our chat transcripts. It would also come up more frequently than other support topics in our public forums.

Knowing the way that things are arranged at WordPress.com, I can verify or deny each of these pieces with different tools. Tag transcripts I could find from our live chat software provider. I could search the Forum, or do a big text scrape. For the knowledge base piece, I can use Google Analytics, since our documentation is all recorded there.

For the best argument, you’ll want to use all of your opportunities to verify – that way you can be as certain as possible that you’re making the right call.

Let’s open Google Analytics. Once you’re in your site or app’s Dashboard, you’re going to see a LOT of information. On the left hand sidebar you’ll see a number of tabs like this:

Screen Shot 2016-05-18 at 8.20.23 PM.png

For most of the work you’ll be doing, the Behavior tab is your friend – much of the rest of the Analytics suite can be useful for support, but would require maybe more digging than we’re ready for, or possibly would require committing additional code in order to track more nuanced behavior.

Since our question is about customers being interested in plugins, one way for us to check our hypothesis would be to see how traffic our support documentation on plugins compares to other support documentation. We know the URL for that document ( https://en.support.wordpress.com/plugins/ ) , so we want to expand Behavior and head into our Site Content > All Pages

Screen Shot 2016-05-18 at 8.25.35 PM.png

From here we’ll get a top ten listing of our most-visited locations, as well as a breakdown of Pageviews, Unique Pageviews, etc. Like so:



OK! Now we’re getting somewhere – I’ve obscured the actual data here, but you can take my word for it that the /plugins/ page is not our most commonly visited support document, with less than 1% of our overall traffic. It is in the top ten, however.

I will note though, that the /com-vs-org/ doc (which describes the offerings of WordPress.com versus self hosted alternative) is highly popular, and for many customers, the difference between WordPress.com and self hosted sites boils down to one thing: access to plugins.

When we take these two documents together, they represent more traffic than every document except /stats/ – but people do so love their Stats. That /plugins/ and /com-vs-org/ taken  together represent the second most visited support document is meaningful, for sure.

We do want to verify that these two documents are in fact related, and what we’re observing here is in fact noteworthy – we can do this in Google Analytics by selecting the Navigation Summary tab at the top, and selecting the /com-vs-org/ page:


Now we’re getting somewhere – in comparing the flow, I see that one of the most common pages folks visit before /com-vs-org/ is /plugins/ – and it’s also one of the most common pages folks visit immediately afterward. I’d take this as sufficient evidence that our hypothesis is supported.WPCOMGA3
It’s highly important that you are careful not to overstate your case – what we can see here is traffic and its flow – we can’t be sure that this is positive or negative, or what impression customers are getting from these documents. It’s clear that there documents are related, and popular, but not necessarily what that means. 

This is why checking several sources and doing a second-level check is important – seeing not only where the traffic totals are, but also how the traffic flows between different pages or stages.

Representing this accurately and researching it thoroughly will help you to state your case accurately. Consider this example, a Mixpanel report of Failed Logins (on the top, in blue) vs. Signed In (successful logins):

Screen Shot 2016-05-18 at 8.40.22 PM.png

Holy moly, we have nearly twice as many failed logins as we do successful ones?! Somebody call the head office, this is a huge problem!

Approach it with curiosity and a desire for verification – imagine, if you fail to logon to an app or service, what’s the first thing you do? You try to log on again, right? Look how this chart changes when we go from “Total” to “Uniques:”

Screen Shot 2016-05-18 at 8.42.10 PM.png

The two have swapped places – yes, 6500 failed logins a day is not great, but it tells a much more measured story, and probably more accurate to your interests.

Answer your questions, but always verify.

The next and final Post in this series will be taking the answers you’ve found, and turning them into convincing arguments. See you soon!






Use the Data You Have: Ask the Right Questions

Once you decide to start leveraging your existing data to unlock the value present in your support unit, the first thing you have to do is start asking questions – not just any questions, but the right questions.

If you haven’t used Google Analytics or Kissmetrics or Mixpanel before, these tools are very powerful, but they can also be overwhelming. The default Google Analytics dashboard has been described as “a dump truck of data.” 

If you go into that jungle without a question, it’ll be very easy to get lost, wander around, and never bring home the treasures that are out there and waiting for you.

The good news is that you already have a ton of really fertile ground for finding and asking great questions. The tougher news is that before you can start sharing real value with others, you have to check your own assumptions and dogma first.

I recommend using your support team’s existing beliefs around your customer base to get started on your quest. Take a minute, think back on the last two or three months, and challenge yourself to identify the big untested beliefs that power your support team. Every team is different, but at Automattic, some of our big ones would be:

  • Our customers want plugins for their sites.
  • Our customers struggle with domains, both purchasing and in their usage.
  • Our customers speak English first and everything else a distant second.
  • Our customers prefer replies from the same person, even if it takes longer to get them.

Once you have a few of these beliefs, the next step is to look at that same set of beliefs, and explicitly ask yourself, is this true?

  • Is it true that our customers want plugins for their sites?
  • Is it true that customers struggle with domains, both purchasing and in their usage?
  • Is it true that our customers speak English first and everything else a distant second?
  • Is it true that our customers prefer replies from the same person, even if it takes longer to get them?

This step is important because it helps you to get in the mindset that you need to be a really great practitioner of data driven support. Whenever someone makes an assertion about your customers or about the way they use the product, your first inclination should be optimistic curiosity.

(In the past I’ve used the term “skepticism” here, but given the more recent usage of that term, I’m getting away from it. It’s become something more negative and more aggressive than its original intent – so optimistic curiosity it is!)

Optimistic curiosity means that you assume best intent, but you’re curious about the grounding of the assertion – does it come from anecdotal information? Does it come from a personal motivation? Is there data to support it? Can we see the data? And so on.

Like the human mind, our questions are best understood by way of behavior – so consider each of your beliefs, no matter how strong, and ask yourself, what measurable behavior would our customers engage in if this belief were true? Let’s go through these examples again:

  • If it is true that our customers want plugins for their site, we would expect that “plugins” would be a top search term in our knowledge base. It would also be a top tag in our chat transcripts. It would also come up more frequently than other support topics in our public forums.
  • If it is true that customers struggle with domains, both purchasing and using them, then we would expect to see a greater incidence of domain related questions than we see for other similarly popular products. We’d also see more traffic to domain related support documentation.
  • If it is true that customers speak English first and everything else a distant second, we would expect to see sites set to English as the distant first in terms of creation rate and traffic. We’d also expect that traffic to our English language support docs would be far greater than other languages.
  • If it is true that our customers prefer support responses from the same person, even if it means waiting longer for them, we’d expect to see higher feedback scores for the products or teams who “own” tickets than the products or teams who do not.

One thing that you’re going to have to get comfortable with, as a data driven support professional, is a little slop in the system.

You’re dealing with humans and human behavior here, so you’ll never be truly certain that you’re right about something – when we start to ask ourselves about behavior that indicates confirmation of a belief or hypothesis, we’re necessarily abstracting away from the actual humans we’re discussing, and in that abstraction we’re accepting a certain amount of slop in exchange for a better understanding.

That understanding comes not from knowing something about your customers, but thinking deeply about what indicators matter. Since you’ll never know, not for sure, you instead have to pick indicators, things that will point to the actual truth even if you can never measure that actual truth.

(To read more about customer research like this, I recommend Just Enough Research by the peerless Erika Hall.)

We’ve moved from untested beliefs into if questions , and developed our hypotheses (our classic if…then statements, above.) In the next Post, we’ll talk about how to actually go find the data around those behaviors. In the third and final Post, we’ll chat about how to turn that data into an argument.

Use the Data You Have: Explanation and Context

Most conference talks are the worst. We can acknowledge that, among ourselves, right?

Many folks don’t properly prepare, they don’t expend any care into their visuals, and they fail to bring anything like the kind of value that they could.

I’m not saying that people who present at conferences are the worst. By and large they’re actually the opposite – they’re some of the best and brightest and most interesting people in an industry, and that’s why they’ve been invited to speak at a conference.

(sometimes they’re even being paid to speak at the conference)

I think it’s more that socially, at least Americans, we conceive of public speaking the same way we conceive of learning mathematics. It’s like a light switch. You’ve got it or you don’t.

“I’m not a math person.”

That’s nonsense of course. But, it’s pervasive, and it unfortunately really sells us short on both ends – folks who have a ton of amazing things to say don’t use their voice because they think it’s simply the way, when it’s more a matter of work, and practice, and preparation.

The other side of the coin are the folks who think they’ve got it, that charisma, and preparation is for squares who don’t have it.

That’s nonsense, too, naturally.

This is a long way of providing context for a series of Posts I’ll be doing over the next few weeks. I’m speaking at SupConf later this month, and I am determined to provide a mountain of value to the folks who have travelled to San Francisco and trusted me with twenty minutes of their time. My talk is called Use The Data You Have. 

It’s about how customer support teams can create value within their companies and for their customers without running experiments or trying new and crazy stuff – just by using the data they already have.

One way I am assuring myself that I can provide some value is by creating the value way ahead of time, in the form of these blog posts, that will serve as a supplement to what I discuss in the talk.

(Don’t worry, they’ll be helpful in their own way as well, I’m not going to keep anything special away from folks who aren’t going to the conference, or are reading this in the future)

In some way this blog series is a way for me to hedge my bets: even if I completely mess up the presentation and look like a total buffoon, I’ll still be able to click through to my final splash slide and cry for redemption; look, look, all hope is not lost!

Plus, this series is going to be somewhat dry, with some screenshots and Google Analytics talk, which is important, but super dry and not at all suited for an in-person conference talk.

Watch this space!