As a video showrunner, you have an important job that entails so much more than just greenlighting some types of content and not others. Your decisions directly impact the success of your brand’s programming, growth, and overall audience perception and loyalty. However, no matter how good your programming is, it can always get better with the right insights, and that’s where this article comes in. We’re looking at how one showrunner in particular helps his company make the best programming decisions possible, so you can hopefully learn something new to apply to your brand, as well!
For just over two years, Evan Bregman has worked as the Director of Programming at the Fullscreen-owned digital entertainment company Rooster Teeth. We had a chance to sit down with Bregman at this year’s RTX in Austin, Texas and discuss everything that makes his programming mind tick. Here’s what we covered during our conversation (click to jump to each topic):
- A typical day at work for Bregman
- Tools he uses to track programming
- How Bregman weighs data from different platforms
- What role data plays in informing content creation
- How he sets programming goals
- Rooster Teeth’s definition of “successful” content
- Using insights to drive customer actions
- What’s changed at Rooster Teeth in terms of data analysis
Pretty much every day is a look in the rearview: how did we do yesterday compared to the same day in the previous week? And sometimes we know exactly why, but a lot of times I will work with my team to figure that out. I have a program analyst on my team as well as just our scheduling manager and we ask, “Why did this happen?” And especially with a new product, we don’t always know the answer quickly, so we may spend a good part of our day tracking it down and then realizing the new app update we released yesterday actually drove a lot of people to watch stuff.
A lot of my day is also being around the content a lot, just from a creative perspective: making sure from a large-scale perspective that, as the people who are really working on the content move forward, they still have broad goals about that content in mind. Along with that, I run the greenlight process, so being able to work with the development team and say, “Here are all the things we want to make and how we need them to perform — what do we have to fill those slots ?” means I drive that into the decision-making process and then tracking that. So my day typically starts at looking back at how did we do at a platform level, but then also what are the individual projects that we are tracking, and then maybe something else tomorrow, and then working on what’s coming next and then obviously looking at a broad scale. Right now our planning is as far out as it has ever been, which I am really proud about, and we are thinking really hard about March of 2019, so when Gen:Lock ends, what do we do after that.
YouTube analytics, of course — you talk to everybody, and that is the gold standard. There’s a lot to say wrong with it, but it is still the gold standard. We use Tubular a lot just to normalize that data as well as to help us navigate the YouTube infrastructure.
You definitely have to separate it, but the good news is we have enough history as far as posting content and enough experience just looking at data on each individual platform, we can be pretty predictive of how the two will interact with each other. The nice thing about setting KPIs is (especially if it’s the very first season of a show) you’re kind of taking an educated guess at the way this is going to work out, and then afterward you have to go back and say, “What’s the story that was told? Were we totally right? Did this perform way differently than we expected but still in a way that is really beneficial for us? Or is this something we should not continue doing?”
It’s pretty rare to get something that’s a total hit and through the roof and knocked out of the park. It’s also pretty rare to have something that fell flat on its face and we realized we did terribly on it. More than anything especially on the first season of a show we are going into it and afterward we are going back saying, “Do we see a way forward in this data? Do we know we can improve certain things where we feel we underperformed and the areas we feel we did very well?”
How do you analyze the videos native to the Rooster Teeth site in particular?
That’s all proprietary, built internally using some of the infrastructure that Fullscreen built for their SVOD service as well as things that are tailored to us. We are a pretty unique business for a lot of different reasons but content can be successful for us and some of our series have KPIs that are specifically tied to things like store purchases, so being able to have a full picture of the way that a series causes users to generally behave in a cohort, whether that is they go to the store and buy stuff related to the series, or a different one for that matter — or it makes them sign up to be First members or something else. All that is important for us so there’s no way to do that without us diving in ourselves and creating it.
Do you ever just look at an overview of both YouTube and Tubular data and think, “Well, this seems to be going well so maybe we should think about doing more of this in the future”?
Totally. Me and a lot of my colleagues as well are big fans of the V7, V30 metrics, because they just kind of normalize the data over time and because it allows you to do that comparison. And even though the definition of a view may be totally different from platform to platform, that’s where I was saying we have enough experience to generally understand how those definitions relate to each other. Then we can set a goal based on that, on how we want them to relate to each other as we change the way we window or as each individual show has its specific goal.
The nice thing about data-driven decision making is the old-school traditional media style of “reading the tea leaves” and doing a lot of focus testing doesn’t really apply to us. Every show is a focus test to us and we get all the data that a traditional focus test would get out every single show so we learn out of everything. Obviously, things don’t live up to what we want them to be sometimes, but there is a takeaway every single time. We have very specific programming verticals each with a strength and a weakness, and right now the goal is especially how do we keep the strength going strong and how do we shore up the weakness.
We had set 12-month targets in a couple of key areas that are important to us based on larger business goals. The specific KPI will be different depending on what the goal is for that show from a business level. Some of our series go on YouTube and there are talk shows that have direct read ads; we measure those very differently than shows that are behind a paywall as our First exclusives. When we greenlight a series, it’s kind of an exercise: does everyone buy into these numbers, these are the goals we’re trying to hit, and then we just kind of track progress against those KPIs.
It’s really different based on the show, and it has to be because each individual series is a business in of itself and when we start a new series, it’s basically like we’re being entrepreneurs opening up a storefront in a city called Rooster Teeth and you’re trying to see if it has legs after the first sale season.
So in some cases we are really focused on viewership and really focused on just driving eyeballs and sometimes conversions to the store from that, or conversions to a direct read ad from that. In other cases, we’re less focused on the number of people who viewed it, and much more focused on just minutes served, like is this something that is reliably getting people to spend time with us and our app? But every single show is different and each individual programming vertical has individualized sub-verticals that are based on individual goals so as a whole, we have a big thriving service.
Sometimes it’s at the series level, like we have a card game Million Dollars, But…. In that case, it’s not complicated: let’s make the series so we can get people to buy the Million Dollars, But… card game. We understand that there is a direct relationship with some of those sales and there is an indirect relationship too, like if we just stopped making Million Dollars, But… (the series) tomorrow would all our sales disappear? No, probably not, but if we don’t have a content strategy around the game, then over time that’s not going to be good news for us.
At the same time, from a broad perspective, my team works really closely with our marketing team to flight out the way that we are using our organic traffic drivers — YouTube being the biggest one — to think through goals. For example, in the 4th quarter the RT store is going to be our big business goal and how can YouTube support that. Other quarters it may be getting people to just come to the website and watch shows. I mean, right now we’ve got Camp Camp and Red vs. Blue that are free to watch in our app; they are not on YouTube. So my big thing right now is can we just get people to download our mobile apps, come to our website, and just watch stuff. And that’s been going really well.
When other shows come around, it’s going to be if we can get people to also then convert to become First members and become paying subscribers. I really work with marketing to make sure we are working to get more specific about the individual values of literally every single button, because you can back out of that and really calculate a floating eCPM and the value you get from each of those individual buttons. We are constantly changing the wording and even now, going through brand exercises, our product just relaunched in April, so we’re trying out different ways to talk about ourselves and just seeing what the community responds to.
Recently, you’ve been pushing viewers to watch content on your proprietary site. How has that gone, actually bringing people over from YouTube?
Really well. I don’t think we ever thought we were going to get the exact same viewership in our app that we got on YouTube, of course not — it’s the second largest search engine in the world. But we’ve been very happy with not only our ability to drive traffic from YouTube and other places, but also just with the stickiness of our new product. People are coming and they’re sticking around so it’s not just blips; it really has been lasting growth, and for us it’s led to a lot of conversations around what’s next.
It’s really nice from my perspective, again thinking about each individual show as a business and all of them coming together to create a portfolio that makes Rooster Teeth run, how every distribution medium calls for a different business model, calls for a different kind of creative. We are now in the enviable position of thinking what is supposed to go on YouTube, what is supposed to be free to watch on our site, what is supposed to be exclusive to First members. And we have a lot of ideas; we know some of that, and as time goes on we are going to continue to experiment and learn more.
We still pretty much make the stuff we want to make, but I think there is a recognition now that we have been able to get more specific, that the content we distribute on our app and the content we distribute on YouTube have to be specific things. Not everything we want to make necessarily belongs in those environments, which is a good thing for us to realize 1) because it’s a better service to our viewers but 2) it also opens up brand new opportunities for us.
Consider our talk show Fan Service, which we did for a couple of seasons on our own app. It did okay, but the better home for that is on CrunchyRoll. The community didn’t bat an eye when it was on our app; the show talks about a bunch of shows that are on CrunchyRoll, so it just makes a ton of sense for it to be there. The show was able to come back bigger and better as a result, and on top of it, we are able to still make a piece of content that we would not have done otherwise because it was not right for our distribution platform. So I think more than anything, we are still making the stuff we want to make, we’re just being more specific about what do we do with that and how we set that up for success, and the data’s informing that.