Category: data

A Civilian at World of Watson Part Three: Philosophy

Back in October I was invited by IBM to attend their World of Watson event in Las Vegas – I wrote a little about it at the time.

Now that I have had some time following the event, I’ve been able to percolate and put my thoughts to paper, as it were. In the interest of you, dear reader, I’ve split these thoughts into three different posts; Technology, Business and Philosophy.

This post is the third and final, talking a little bit about Philosophy. You can find my first post, discussing the Technology and my experience of it, here. The second post, discussing the Business of IBM and the conference, here.

Continue reading “A Civilian at World of Watson Part Three: Philosophy”

A Civilian at World of Watson Part Two: Business

Back in October I was invited by IBM to attend their World of Watson event in Las Vegas – I wrote a little about it at the time.

Now that I have had some time following the event, I’ve been able to percolate and put my thoughts to paper, as it were. In the interest of you, dear reader, I’m splitting these thoughts into three different posts; Technology, Business and Philosophy. Note that there are no bright lines here: I’m sure to touch on each of the three topics in all three posts.

(I predict #1 and #2 will far outshine #3 in terms of traffic – such is life, friends!)

This post is the second, talking a little bit about Business. You can find my first post, discussing the Technology and my experience of it, here.
Continue reading “A Civilian at World of Watson Part Two: Business”

Metrics, Means, and Maps

As a younger man, I spent a lot of time reading and discussing philosophy.

In the end, I was most attracted to modern moral theorists like Rawls and Nozick, but like all Philosophy majors at the State University of New York at Binghamton, I spent some time with all of the greats: Plato, Aristotle, Kant, Descartes, Marcuse, Arendt, and so forth.

(In fact, in the forward of Anarchy, State and Utopia, Nozick describes what I think is the most perfect description of all professional academia, not just Philosophy. I’m away from my copy, but I’ll post the passage when I get home!) edit: I gave it its own Post!

I’m bringing this up because one of my least favorite philosophers to read was Immanuel Kant. I struggled with Kant, like I suspect many 20 year olds do, as his writing is so incredibly dense, and translated from the original German. One piece of his moral philosophy that stands with me is this: to behave morally, a moral agent must treat other humans always as ends in themselves, and never as means.

To be more philosophically precise, Immanuel says never to treat other humans merely as means, but always as ends as well.  So, it’s not necessarily immoral to treat another human as a means, so long as you keep them in mind as an end also. It’s a tricky bit that’s easy to forget. Kant, he’s dense.

One thing that we need to bear in mind, whatever department we’re working in, is that our metrics are necessarily abstractions, a means to a larger end. In this way our mindset needs to be like Kant’s – some things are ends, some things are means, and we should be intentional about which is which, and remind ourselves that the distinction is important.

A quote that came up a number of times at the Growth Hackers convention this year was this: “Be careful what you optimize for,” and that, too, points at what I’m getting at here.

Our metrics, our measurable indicators of success, must necessarily be abstractions from real life. 

By this, I mean, reducing churn by 10% is only a means to a larger end, and has to be considered in that larger context. What’s the real reason? Why do you, personally and as an organization, want to reduce churn? Maybe it’s because you believe you have a product that can genuinely make peoples’ lives better, so the more folks who use it, longer, the better off they’ll be. That’s great! Maybe it’s to make more money – that’s OK too. 

In either of these cases, churn reduction is itself only a means toward a larger end. Success with this metric points to a larger success, something that you’re maybe not equipped to measure, something like Customer Happiness or Success of the Business. We need to keep this in mind.

Another quote that’s on my mind a lot these days: “The map is not the territory.

Our metrics are only maps upon which we build our assumptions and beliefs – the underlying terrain, the real territory of your customers and your business, is far more complex, far more nuanced. Remember that we use metrics because they are abstractions, because they take our complex world that is impossible to understand all at once, and break it into easier-to-understand chunks.

Our metrics are by design not the whole truth. They’re reductive because they must be – because only by reducing a complex concept can we hope to make meaningful decisions. If our metric were the whole truth, if the map were a perfectly accurate representation of the whole territory, it would be perfectly useless.

Measuring our work, and our companies, and our success or lack of success, is absolutely vital to the success of any enterprise in 2016. Choosing the right metrics, and bearing in mind that our metrics only represent one part of the truth, is the hard part.

 

You’re Already Interviewing Your Customers

Let’s start with a story!

At Automattic, we’re lucky enough to have some pretty sophisticated internal tracking and analysis tools. I was recently involved in a conversation with my friend and colleague Martin, about a particular slice of our customer base, whose churn is higher than we would have expected.

One of the ingredients for this particular group of customers was that they had, at some point in the seven days before leaving our services, interacted with our Happiness Engineers via our live chat support offering. Given the tools at our disposal, we were able to pull together a list of all of these customers – and with the churn rate being what it was, and the total userbase for that product what it was, the list was not terrifically long. Double digits.

Some of you out there know this story, right? What better way to find out what is going on with your customers (or former customers) than asking them outright? Put together some post-churn interviews, offer an Amazon gift card, learn something new and helpful about your product or service. This is a pretty standard flow for researchers – start with Big Data to identify a focus spot, then focus in with more quantitative methods, interviews, surveys, what I think of as Small Data.

In this case, rather than jump to the usual move, and at Martin’s suggestion, I pulled up all of the chat transcripts, and read through them, categorizing them along obvious lines, pulling out noteworthy quotes and common understandings (and misunderstandings!) – treating these last live chats with churned customers like they were transcribed interviews, because in a real way, that’s what they are.

I was really surprised how insightful and interesting these live chat sessions were, especially when read back-to-back-to-back like that. In fact, I did not even feel the need to follow up with any of the customers, the picture was clear enough from what they’d already communicated with us. I was honestly floored by this, and left wondering: how much good stuff is already in these transcripts? 

Moving forward, I’m including customer email and live chat review as an integral part of any user cohort research that I do – it will allow me to come to the interviews three steps ahead, with far better questions in mind, and a much sharper understanding of what their experience might have been like.

Especially with robust data slicing tools, being able to cut down through verticals, cohorts and purchase levels means that I’ll be able to see a ton of useful, relevant conversations with customers similar to those I’m looking to learn more about.

This is also the case with you and your customers.

Even if you don’t have a user research team, or even one researcher, your support team is interviewing your customers every day. Even without data slicing tools, you can do something as simple as a full-text search on your last month of email interactions and get something close to what you’re looking to learn.

If you enjoy a support tool that has a taxonomy system or plugs into your existing verticals and cohorts, all the better.

This Small Data on your customers, these conversations, already exist. You don’t need to generate new information, you don’t need to sign up for third party user testing.

You’ve heard me say it before, folks – there’s value in the data you have. Use it!

 

 

 

 

Full SupConf Video for Use the Data You Have

Hey folks! I’m really happy to share with you the full video of my recent talk at SupConf 2016, where I gave a talk on leveraging your existing data to build value from your support organization – here it is!

I also created a supplementary Page with a three-parter on how to execute the ideas I present in the talk – you can find that at https://s12k.com/supconf/ .

SupConf East is coming soon – as a speaker and an attendee, I cannot recommend it highly enough! You can get updates on the next SupConf event by signing up for the mailing list here.