Change Agent: NPR One: Learning from the Audience
by Nick DePrey and Sara Sarasohn, NPR
This article first appeared in the Fall 2015 issue of Change Agent
NPR One is the new way to listen to public radio. In smartphone apps, connected cars and TVs, you can hear a handpicked selection of local and national stories, personalized just for you. You can timeshift the news and discover podcasts you didn’t know you’d love.
It is also an unprecedented source of information about the way that the audience listens. Until now, we’ve never really known how people hear stories. On the radio, we measure how many people listen in an average quarter hour (AQH), but we have no idea if they heard that great interview about the Confederate flag yesterday. Apple counts downloads for podcasters, but they don’t say if those downloads have actually been listened to. With NPR One, we can know it all. Our data describes each person’s relationship with every story they hear.
Here’s what’s really amazing: By watching the audience data, the NPR One staff can respond to listeners in new ways.
Get all the good stuff. On the radio, the convention has always been to put the most important news story at the top of the hour. “Lead with the lead!” we say, chomping on our metaphorical cigars in our newsrooms. The latest developments always go at the top of the hour. But that’s based on a lie we tell ourselves about how the audience hears our work. They don’t turn on the radio at the top of the hour. The first thing they hear could be anything. Leading with the lead at the top of the show does not mean more people will get the thing we think is the priority.
On NPR One, everyone starts at the beginning, with a three minute national newscast. After that, we know how long they listen. Right now, average session time is around 30 minutes. That is a crucial piece of information. We know that all of the stories in the first, say, 20 minutes of the experience are going to touch the majority of our listeners. We have 20 whole minutes to craft a complete experience.
When we have a story that we think is essential – important news, water-cooler must-know, just a lovely piece of radio – we can make sure that most of the audience will have access to it, just by getting it into the first 20 minutes of the lineup. That one piece of data – the average session time – is critical to how our editors make story choices. It makes NPR One more precisely curated than the radio.
The newscast machine. A few months after launch, we looked at when our audience was leaving, and came up with this graph:
That huge spike is 29% (?) of our sessions ending when the national newscast is over. At first, we were delighted because the audience was using NPR One in a way we had not anticipated. One friend said, “I do that all the time, listen just to the newscast to make sure the world is still turning.”
It was also a little disappointing. People were turning on NPR One, listening to the newscast, then turning it off. They didn’t get to any of the more in-depth feature reporting or fascinating podcasts we had for them.
What could we do to keep them listening longer? We decided to see if we could tweak the first story after the newscast to tempt the audience into staying, even when they didn’t mean to. This was moving even farther away from “lead with the lead,” but we decided it would be worth it. After all, we could keep the lead anywhere inside that 20 minutes and people would still hear it, even if it was not the first thing.
The traditional lead story in a news radio show is the most important story of the day, regardless of audience interest. We expanded the definition of an “NPR One lead” to be anything really interesting and news-related. These stories, we hoped, would lure some of those newscast-machine listeners into staying longer. There was a lot of experimentation, and sometimes we probably went too far afield from the news. We watched the audience stick around more and more during the first story and over the course of weeks, we got that newscast machine spike down closer to 23% (?).
Better writing for the radio. One of the greatest opportunities the NPR One data provides is about individual listening to individual stories. We know, for example, how many people skip and when in the story they skip. This skip data is unprecedented. It tells us when the audience loses patience with us.
There was an obvious first thing to analyze with the skip data: the host intro to stories. In the newsroom, the host intro has been the subject of debates, trainings, revisions and dismissals over the last few decades. There was nothing else we wanted to turn to when we wanted to analyze storytelling techniques with NPR One skip data. Starting with hundreds of intros from the month of March, we identified 33 intros with very low skip rates. Experienced NPR editors listened to all of them and identified two clear patterns.
- The great majority of them were 22 seconds or shorter.
- The great majority of them began with a clear topical statement (“The Transportation Security Administration is coming under fire for a method it uses to spot possible terrorists.“) even when they could have begun with the latest development (“… the American Civil Liberties Union has filed a lawsuit.”)
What’s key about the second finding is that, at its heart, it’s about not leading with the lead. The most interesting thing, not the most recent thing, is the way to hook the audience at the beginning of an intro. This is the same thing we saw with the lead story after the newscast. Listeners stuck around when we focused on placing the most interesting story at the top, not the one about the most recent detail.
This led us to think more about our values as journalists. We see ourselves as the inheritors of the tradition of the cigar-chewing newsman banging on his typewriter with two fingers. His value was the up-to-the-minute news lead based on what happened today. The audience seems to be saying, through the NPR One data, that they care more about economic changes fracking brings … than how many signatures the ballot measure about fracking got.
We have to consider that perhaps the value of our profession is not the same as the value of our audience. When we talk to them, we might consider more how they approach a news story. This is not an easy lesson for us, because it runs counter to the lore of our profession. It comes to us from a new source – data – that we are not used to listening to as radio journalists.
NPR journalists are open to learning from NPR One data. The specific findings about writing for intros were presented to more than 200 public radio journalists. It has been one of the most talked-about things in the NPR newsroom for the last couple of months. It also generated real changes for reporters who write intros and hosts who read them on the air. Changes in the values of our journalism have been discussed as well, but any change there will be slower.
Talk directly to the audience. This is not unique to NPR One, but we have found that feedback from our users on Facebook and Twitter has helped us identify bugs and outages of our service. We built a sleep timer because of social media requests. We’re going to focus on improving our data handling because it’s a weakness pointed out by our listeners. We have also had actual phone conversations with some of our audience.
Because we are public radio, we say, “Beloved is our business.” People love us so much they give our member stations money even when they don’t have to. We wanted to understand why our listeners donate, so we asked them. These long conversations with 12 listeners explored their listening habits, the connection they feel to public radio, and what motivates them to donate. We will use their comments as we try to make a new kind of membership connection that’s not based on the radio.
Another big project we have is around podcasts. On conventional podcast apps, you subscribe or unsubscribe to a podcast. That’s easy to handle, technically: it’s on or it’s off. That’s not how people feel about their shows. They love a few, will listen to a bunch, and might hate one or two. We built a scoring system and an algorithm that we hoped would mirror how listeners feel about their podcasts.
It seemed to make sense, but there was only one way we could think of to figure out if it really worked: we had to ask people. So we called about 15 of our listeners, explained the process, and read them their scores. They were delighted to talk to us and fascinated by our work. They loved getting a peek under the hood of the NPR One algorithm, and the idea that they could contribute to how it improves. They also confirmed that we were on target in figuring out what they liked and didn’t like.
We’ve been looking at web pages for 20 years and keep finding new ways of quantifying audience behavior – from hits to clickthroughs to time spent on page. NPR One is less than a year old. We are just beginning to learn to use data to recommend podcasts, improve production techniques, and introduce our audience to more great public radio they didn’t know they were going to love.