At Spirited Media, I coordinated the launch and execution of membership programs for Billy Penn, Denverite and The Incline. We learned a lot, fast, and built a suite of tools to help us measure success and make data-driven decisions. A year ago, Spirited broke up, and the newsrooms found new homes. I’ve continued to work with Billy Penn and Denverite, and we’re super excited to share what we built together.

Memberkit isn’t a proper software project, it’s a bunch of spreadsheets, a bit of code, and some techniques for how to track your readers’ habits and ask for money more effectively. But they hang together in a tidy way, and it’s fun for projects to have a name!

Everything here is field-tested and free for you to copy and use in your newsroom. Even if our tools aren’t a perfect fit for your situation, I’m hopeful that Memberkit can help you think about how to execute a data-driven strategy for membership.

Why we built these tools

How do you inform data-driven decisions? IMO, it can be reduced to this:

1. Ask good questions.
2. Get the data to answer those questions.
3. Talk about those answers, often.

Unfortunately, the most common tools newsrooms use for analytics fail at all three steps. Why?

They answer the wrong questions: Google Analytics, MailChimp, Chartbeat, Parsely, etc. all deliver tons of data, but they weren’t built to support a member-powered business model. MailChimp was designed to be a marketing tool, not a daily newsletter authoring machine.

It’s really fun to watch the Chartbeat needle spike, but does it tell you if your business model is succeeding? If your business is selling impressions to advertisers, then maybe. But membership is about loyalty, not traffic. You can make these tools answer good questions, but they don’t come that way out-of-the-box.

(Also, FWIW, I’m not saying these tools are, um, evil… they’re just tools. But there is a real danger in putting the traffic needle on the newsroom television—when you celebrate the wrong numbers, you create the wrong incentives. Like the person said, “What gets measured gets done!”)

Plus, it’s a chore for someone to extract the data (copy-pasting from Google Analytics, exporting a CSV file from Mailchimp) and present your findings. You’ve gotta get in the habit of looking at your numbers, as a team, as frequently as possible. And if that’s a lot of work, you’ll fall off the wagon. It’s gotta be set-it-and-forget-it!

And that’s why people build dashboards. But most dashboards stink, and it took me a long time to finally realize why: They’re usually a snapshot in time. “At this moment, 200 people are reading our website” or “Last month we gained 10 members, 100 newsletter subscribers, and 10000 readers. 1000 of those readers were recurring.” Etc., etc.

A dashboard is alright if you quickly need to know your current status: Am I exceeding the speed limit? Is the pollen count high today? Who is winning the baseball game? What posts are currently popular on our website? But if I’m trying to track my progress toward a goal over time, a dashboard is barely helpful.

To inform data-driven decisions, you don’t need a dashboard. You need a fitness tracker.

Like a dashboard, a fitness tracker shows you your current status, but it also shows you how you got here. Done well, it can show your velocity (“How many new members do we convert every month?”) and your acceleration (“Are we converting more people every month?”). A data-driven team knows not just how well they’re doing, but if they’re getting better (or worse) at doing it! And then we adjust our strategy to keep improving.

So, without further ado, here are many of the questions we asked, and the fitness trackers we built to answer them. (Plus a few techniques that use our tracking data to convert readers to members!)

How many people read us frequently?

As far as I know, the most important thing to membership-powered newsrooms is the loyalty of our readers. Lots of good stuff has been written about the value of loyalty, but how do you measure it?

The real question I want answered is “Are we getting better at encouraging frequent reading?” So, first I need to track how many people read us frequently. And second, to get a sense of our progress, I want to know that number month-by-month. How many people read more than five articles this month, last month, two months ago, etc.

A simple measure would be to track session count. That’s relatively easy to get from Google Analytics, just go to Audience > Behavior > Frequency & Recency, but unfortunately these numbers aren’t what we want. The session count in Google Analytics accumulates over time, which means someone might have had their first session three months ago, and their second session this month. So when I look at this month’s Frequency & Recency report, the number of people in the 2-sessions bucket is everyone who had their second session this month—not the number of people who had two sessions during this month.

To get real numbers, you’ve got to do a little work. We created a custom dimension to track reading frequency, and wrote a bit of code that counts the number of articles read per month: at least 1 article, at least 2–3 articles, 4–5 articles, 6–7, etc., etc.

The 6–7 count matters most—we use that as a shorthand for the maximum number of people who might become members. If only a thousand people read more than 5 articles per month, then we’re not likely to have more than a thousand members.

Want to use this report? Here’s the spreadsheet. You’re gonna need to implement a custom dimension in Google Analytics. Here’s the writeup about how we did that.

Are we growing local readership?

There are a lot of ways to optimize your membership program—improving your conversion rates, writing better appeals, etc., etc.—but once we knew the number of people who read us frequently, we realized that our big problem wasn’t conversions, it was volume. We needed far more readers if we were going to support our newsrooms via membership.

So, we decided to invest in marketing. Specifically, advertisements on Facebook and Instagram. Eventually we hoped to see more frequent readers, but the leading indicator of success for this effort was the number of new readers visiting us each week. And since we cover cities, we only care about local traffic—a successful ad, or a big, viral story only matters if it brings in more locals.

In addition, since we were advertising on Facebook, we created a special version of the report that measured only local readers who came to us that way. And since readership patterns are cyclical (May and October are always big, because elections), we look at this data month-by-month and year-on-year.

Want to use this report? If you’re a city-wide publication, check out the spreadsheet for cities and metro areas. If you’re a state-wide publication, we’ve got another spreadsheet for in-state vs. out-of-state readers. Want to know how the spreadsheet works? Here’s the writeup.

What articles do members and subscribers read?

Growing readership is great, and a hit story might attract a lot of new, local readers, but what do our most engaged audiences care about? Similar to how we track reading frequency, we also set up custom dimensions to track the reading habits of our newsletter subscribers, donors, and members.

Looking at last month’s numbers, some stories are universally popular (Who can deny a headline like “What smells where in Denver”?), and the more geeky stories about housing, history, and urban planning are favorites among Denverite members and newsletter subscribers. This isn’t particularly surprising, but it’s a nice reminder that though the geeky stuff might not get the most total pageviews, it resonates with our most valuable audiences—if we want to get better at membership, maybe we should write more stories like this!.

Want to use this report? Super! This one’s more of a demo—we never got around to making it fancy. To use it out of the box, you’ll need a custom dimension in Google Analytics. Here’s the writeup about how we did that.

Don’t have any custom dimensions set up? We’ve got you covered. Check out this report that uses Google’s built-in age-bucket dimension to compare what people of different ages read.

We’ve also got a version of this report that uses our reading frequency custom dimension. Across the board, the coronavirus map was popular among all reading groups, but watch as Hannah Callowhill Penn comes from out of nowhere to take the top spot among really frequent readers!

Nerd bonus! Open the script editor to see how we’re color-coding the articles—hashing and bitwise operations, FTW!

What’s the best way to convert a particular reader?

We talk a lot about “the funnel” in the membership business. The most common interpretation goes something like this:

1. You become aware of us, via a friend’s post on Facebook
2. You like what you see and decide to get our newsletter
3. You think the newsletter is awesome, and you decide to donate money

Ideally, the user experience of your website and emails will lead people down that path. Following that simple formula, we will ask you to subscribe when you’re reading an article, and we ask you for money in the newsletter.

But reality is not that simple. Our readers are different. Some know us well, others are reading their very first article. And there are lots of people who read our website all the time, but have absolutely no interest in getting our newsletter.

So, there’s not just one path to giving. And, as such, you probably shouldn’t be delivering the exact same user experience to all readers. How might we adjust the UX so that we’re delivering the right message to the right people?

Turns out it’s not that complicated. We use the very same rig that tracks reading frequency and the habits of members and newsletter subscribers. If you’re reading an article on our website, and…

• We know you’re a new reader, we ask you to subscribe.
• We know you’re a newsletter subscriber, we ask you to donate.
• We know you’re not a newsletter subscriber, but that you also read us frequently, we stop trying to get you to subscribe, and just ask you to donate.
• Etc, etc.

Want to deliver the right prompts to the right people? We’ve documented our motivations, our code, and even our backend user interface. Copy our technique!

How many people actually read our newsletter?

Newsletter numbers are tricky. You can track the number of subscribers, open and click rates, unsubscribes, etc., etc. And to confound matters, we don’t just send one kind of email. Our daily newsletter makes up the bulk of what we send, but we also send automated membership appeals, special membership campaigns, invitations to events, breaking news alerts and more.

It’s useful to occasionally dig into the performance of a particular email. If your click rates are low during your summer membership campaign, I’d want to look at those numbers really closely. Maybe run a multivariate test (the fancy name for what we know as A/B tests) to try other approaches. But it’d be seriously time consuming to be that obsessed every day, or every week.

And we really want numbers that we can look at, at a glance, every week. So we need to do three things:

1. Determine the most useful big numbers that we can look at on a weekly basis.
2. Separate those numbers for our different kinds of emails.
3. Automate the hell out of it.

The biggest useful number for our daily newsletter is “total weekly opens”. That is, the total number of times our daily newsletter was opened, every week. It’s basically a combination of the number of subscribers and the open rate. And if it’s going up every week, we’re happy.

Use our 6-month campaign performance report to track total weekly opens, and a handful of other metrics, for our newsletter and other email types. This one uses the Mailchimp API, so you’ll need that, and you’ll need to categorize your emails into folders. Want to know how this works? Here’s the writeup.

How good are we at converting newsletter subscribers into members, fast?

Our understanding (learned from other smart folks, and backed up by our observations) is that a reader’s proclivity to donate decreases quickly over time. That might seem counterintuitive: Wouldn’t someone who has found your product valuable for several months decide to give? Yes, some will, but many have grown used to not paying, and blind to your requests to give.

So, once you’ve got somebody interested, you want to get them paying, fast. When someone signs up for our newsletter, we immediately kick off an automated series of emails (known in the industry as a “drip campaign”—maybe not a new idea for you, but it was new to me!) that, among other things, ask our new subscriber for their hard-earned cash.

Many people will tell you to slowly build up to the $$$ ask over a series of emails. First, you start by introducing yourselves, talk about the value you provide, the things you cover, etc., etc., and after several emails, you eventually start explicitly asking for money.

But we found the opposite to be true. The longer you wait, the less likely people are to click the link to donate. The first and second emails in our drip campaigns always performed the best—the highest open rates, the highest click rates, and the highest donation rates. So, yeah, we want to ask early and get folks to donate as quickly as possible.

Every week, we look at the percentage of people who decided to donate within their first 30 days of being a newsletter subscriber. I want to see that number be at least a couple percent, hopefully more. If you’re planning to make adjustments to the language in your email welcome series, or donation prompts in the newsletter, set up this report first, and let it run for at least a few weeks. That way you’ll have a baseline of data, and when you change things, you’ll be able to see if your change had a positive or negative impact.

Wanna use this report? If you use Mailchimp for your newsletter, then we’ve got you covered! The report will be relatively easy to adapt, but you may need to call a programmer. Don’t have a writeup for this one, but the docs are pretty thorough.

Is donating hard? Can we make it easier?

From a user’s perspective, the donation flow looks something like this:

1. Click a “donate” link/button in our newsletter or on our website.
2. Read our “support our work” page, select the donation options like dollar value and frequency, and click to proceed.
3. Fill out payment details like your name, email, and credit card info, and click to donate.
4. See a payment confirmation page, click to go back to our website.

There are a lot of points in your donation flow when a user might bail out. Maybe the messaging on the “support our work” page isn’t compelling, or the donation options are confusing. Or perhaps we’re asking for too much information on the payment details page. (Why do they want my mailing address?)

Anyway, if we want to optimize this workflow, we first must measure it. And that’s where Google Analytics’ “goals” function comes in handy. If you’re like us and use a third-party for payment processing, you’ll need to set up those goals to work on your website and on your vendor’s website.

Luckily, this is totally doable. And, FWIW, the same concepts apply to other third-party vendors, like an event registration system. So you can also know if your event signups are failing on the vendor side.

Want to measure your donation flow? Would you like to know if users from different sources (e.g. your website vs. a link on Facebook) behave differently? Curious if your users are dropping off when they hit your vendor’s checkout page? Here’s a writeup explaining how you’d find out!

Alright! Ready to dig in?

That’s all I’ve got for you today! Here’s a folder with all the docs. Everything is Creative Commons-licensed, and free for you to copy, use and improve! Go to town! We’d love to hear what you do with Memberkit, so please stay in touch! If you wanna know a little more about the genesis of Memberkit, read on!

Who are you?

Hey! I’m Brian Boyer. I used to be the VP of Product and People at Spirited Media. We were the home to The Incline, Denverite and Billy Penn. In 2019, Spirited was broken up. But—and this is where it gets weird—we had just completed an accelerator program and had received a grant to help our membership programs. So, I teamed up with Denverite and Billy Penn to just do the project anyway.

During our time at Spirited (and accelerated by the accelerator!) we implemented many tools to help us measure the success of our membership efforts. We’re calling those tools Memberkit, and we’re geeked to share what we learned!

Much of the designs and code in the linked docs were originally created by the Spirited Media product team: Russell Heimlich, Chris Montgomery, Livia Labate and myself. Huge thanks to Danya Henninger and Dave Burdick, my colleagues at Billy Penn and Denverite, respectively, for making it all available to share with the world.

The code in the reports (when there is code) was written by me. So if that stuff is broken, it’s nobody’s fault but mine.

• Share on Twitter
• Share on Facebook