A Few Good Questions

There’s been some press recently about using a single “key metric” to measure success, like Medium’s “total time reading.” We don’t have one “key metric” at the Times. Instead, we’re gathering a more complex and dynamic picture of readers by asking a few key questions you can try out too.

What are people reading? In a news organization, you can break that down in several ways. By desk (National, Science, Sports), subject (elections, surveillance/privacy, pro football), or publication time (breaking or current news, past enterprise stories, older stories). Perhaps some readers only come for new Sports stories or scores and that’s it. Perhaps others come for Sports and then navigate to other sections.

There can be anomalies in reading patterns, and those are useful. Finding higher activity around older stories can lead you to other questions—like, why is a story suddenly gaining traction again, is it related to a current news story? With those questions in mind, you can decide how to promote or contextualize these resurgent stories better.

When do people visit your site or apps? Overall trends might be in the morning, before they leave for work, or at lunchtime on break. Do morning visitors repeat throughout the day? What percentage of visitors are new overall?

How are readers interacting with your content? What percentage use a desktop or laptop computer compared to smartphones or tablets? Higher mobile usage should help you make a case for mobile-first or responsive design—a hot topic in news orgs these days. How can you better serve readers across all devices?

Who are your readers? This is a big one. Asking this question is essential to a more nuanced view of your audience. My colleague James Robinson refers to this as audience segmentation—which replaces the traditional view of the singular newspaper subscriber with the view that different segments of the population visit for varying reasons. Some things you can measure here are the basics, like gender, age, and location. How many of your visitors are logged in and how many are anonymous? How many are subscribers? Do subscribers of the print edition engage digitally in similar patterns as those who do not? Do you have readers who only interact with your site via mobile or is it more mixed?

Geographically, are you getting more traffic from outside the region than you expected? How might this change the way you present coverage to a global audience?

What’s Next?

As you make progress answering questions like the ones I listed above you’ll probably encounter some surprises about both your audience and the newsroom you sit in. Is a big story getting the number of eyes everyone expected or hoped it would? Metrics can help you understand why. I ran into this last year—an investigative piece didn’t get the traffic we’d typically expect, even though it was given special design attention for the site. It turns out that the readers most likely to find it interesting probably missed it—the headline didn’t do a good job of conveying what the story was about. In other cases, we’d suddenly see sharp drops in traffic to stories that were otherwise really successful, only to find out these stories stopped getting promoted due to a routine scheduled refresh of section fronts.

Obviously readers can only engage with the content they can find. These examples led us to further measure how many visitors see links to articles, what kind of link—a simple text link, with or without a photo, etc.—and where it appeared. If stories are falling flat, it’s good to be able to rule out faulty promotion and focus on why the subject matter or presentation might not be resonating.

Hopefully you can also see how the numbers are just part of the story here—you can track every little thing each reader does on your site and have detailed charts showing trends based on this, but you’d still be missing the big picture. I’ve learned that data can’t 100% answer questions about what comes down to human behavior. Data can only suggest trends and help inform the conclusions you’ll draw using your own experience in the newsroom and as a consumer of news yourself. Analytics data should also challenge existing assumptions or customs in your news publication culture. Your website and apps don’t have a print deadline like your publication’s paper version (if you even have one)—why should your news coverage roll out like the paper does? If you don’t use the traditional print deadlines, what should shape decisions to publish content? How long should articles be promoted for, and where?

Conversations my colleagues and I have had with editors and producers throughout the newsroom have helped us figure out how to better present trends to inform editorial decisions. Sometimes, it turns out graphs—which require a certain level of contextual knowledge and time to interpret—aren’t the best way, especially when quick decision making is of the essence. To that end, we’ve started experimenting with generating quick alert emails to staff with things like, “It looks like the article [link] that was last on the homepage on [date] is going viral on social media and getting a lot of traffic. Consider linking to [associated article 1] or [associated interactive 2].” We’re also excited about using chat bots to allow the newsroom to quickly query metrics.

I’m fascinated with analytics and I’ve seen how much there is to learn about making a better news experience for readers. That said, you can’t forget that readers are dynamic human beings who can only be partially understood with metrics and analysis. In addition to looking at several measurements and researching the context of each, as in the example above, there are other ways to get to know your audience. As a human being yourself, you already should know one: talking to people. Qualitative data is even more important than quantitative, and you can get it via interviews, polls, surveys, focus groups, and simply, discussions. The point of analytics isn’t just numbers, it’s about meeting your audience in new ways.