YuMe teamed with IPG Media Lab to do an extremely interesting study on ad effectiveness which included biometric tracking for emotional response, attention. With bio-feedback and eye-tracking, I do believe it might be a one of a kind study for online advertising, plus, I’m a science nerd of Peter Parker proportions, so I’m totally excited to dive into it!

So all that fancy schmancy technology they used should have given some pretty interesting insight into what it is people are doing, how they’re consuming, how they react and how they look at online advertising. But, as with all studies, I first want you to see how they collected the data, so here it is.


The research was conducted in-house at IPG: In-Lab Test Across 4 Screens: Connected TV,
Linear TV, Mobile, PC. It included adults 18+, familiar with at least one of the screens and they intentionally recruited tourists (n=147). It was done 5/9/2012 – 5/14/2012The participants were assigned to screens, ads and content types by a survey which indicated which they owned/used. Participants watched pre-recorded videos on designated screen(s), then answered follow-up survey about media. A final survey was given for unaided and aided recall and they were then re-exposed to an ad. Ad load/frequency was designed to match the typical viewer experience.

The study aimed at tracking three metrics: attention, excitement and recall. It utilized eye-tracking software from tobii and biometric bracelets from affectiva. Demographics are broken down below:

Now, if we use their n=147 and extrapolate out to the entire U.S. Population (roughly 313 million) the margin of error is pretty large, 8.08% and it’s the same at 200 million. So remember that because I think it’s a major flaw in the report.

Connected TV Tops Tradition TV, but Not PC, in Ad Recall

An interesting bit of info that came out of their study was that ad recall on regular TV is fairly low, just 27%, while even on a connected TV it rises to 38%. I have to believe that it might be due to the potential for interaction? Viewers generally know that TV ads will be fairly static as in there’s not going to be some sort of call-to-action that results in getting a store location, coupon, etc.  Then again, the pool of participants in the connected TV portion of the study was much lower than the TV part, which could create some statistical anomalies.

TV does elicit higher than average attention and excitement levels even though ad recall is fairly low.

So while your ads on TV might create an immediate reaction in the viewers, it seems there’s little lasting impact on them.

PC Tops Them all

In terms of recall, PC topped both TVs and mobile, yet it seems to me that tablets are conspicuous by their absence in the study. PC scored a 43% unaided video ad recall rate, but about the second lowest amount of excitement, at just 5% of total time, topping only mobile (4%). Linear TV managed 7% while connected TV got 8%. Clearly, ‘excitement’ isn’t something associated with watching video. They must have been watching librarians catalog books or a dog show or something. Seriously, video is supposed to elicit emotion, but according to the study, it seems like their respondents were mostly sleeping through it all.

Since we don’t know what they were tracking with the bracelets as “excitement” nor what the content was that the participants were watching, it’s hard to judge what exactly this means. Has video viewing become a zombie-like activity? Are we simply lying back and letting the images and sounds wash over us without being affected by it? What kind of impact does it have on advertising effectiveness? They say that excitement and attention are high on TV, but then ad recall is quite low which doesn’t bode well for advertising on TVs.

Clutter? In Linear TV?

Perhaps it’s just that ad blindness is more prevalent that suspected. When we look at their findings in regard to ad clutter, we see that it could be the cause of the low recall numbers on TV.

The lower ad times resulted in better unaided recall. I wonder if, during the study, participants were allowed to wander off to the kitchen or bathroom during TV commercial breaks. That’s really what they have become haven’t they? Oh a commercial break, that’s 3-5 minutes I’ve got time to make a sandwich, grab some chips or use the toilet…

Creative is the Key?

It seems that being in the 8-10% range has a drastic positive impact on brand recall. I have to also suspect that it’s the difference between a couple short ads versus a stack of longer ones. Traditionally, ads online have been in the 15-30 second range. I find that if I’m watching Hulu, and the ad break says 60 seconds, I just flip over to a second screen. If it’s 15-30 seconds, I generally just ignore the ad as it plays. Of course, there are really only 3 ads on Hulu any given day. Perhaps that another factor in the unaided recall, ads on Hulu are shown more repetitively than they are on TV. To the point of annoyance in fact for me. I’m tired of seeing Geico ads, especially since I’ve told their alleged ‘ad tailor’ that I don’t need insurance. I don’t even own a car in fact! Maybe it’s the content of the ads that’s helping recall? Then again, maybe not. Look at these results.

They say in the title “creative quality” but the results are based on ad likeability. I’m not sure those are exactly the same thing. Additionally, the results aren’t all that linear. Retail had quite low recall but quite high likeability. Auto had massive recall with a similar likeability, etc.

Missing the Mark or Burying the Lead?

YuMe and IPG Media Lab summarized the findings as such:

  1. Clutter free environments, regardless of screen size, are a good value
  2. Advertisers without media budgets for high gross rating point (GRP) TV campaigns should consider moving to screens with less ad clutter to ensure campaign breakthrough
  3. Creative testing is strongly recommended whenever possible – and digital video is a great platform for testing different ad creatives
  4. Since prime-time for linear and connected TV is also the peak time for second-screen usage, consider buying placements across devices during this day part when planning for duplication

Personally, I think their assumptions are shaky. The sample number was extremely low to begin with. They seem to not have taken factors into account like personal preference of content. In the results they show that Hip Hop generated above average attention but Syfy was well below average in attention and engagement. But were the viewers given a choice of what to watch? It’s not stated anywhere in the report.

They state that ad clutter created lower recall and attention, but there’s no ‘clutter’ in linear TV. You’ve either got an ad showing or content showing. They say screen size doesn’t matter, but then completely missed a size category in tablets. They also seem to state, in number 3, that you can do A/B testing with digital video and then transfer that creative to TV, but, that wouldn’t always work, and we all know that. Those are going to be different viewing audiences, with different expectations and different attention spans. Sure, there will be some overlap, but to blanket it all with that statement seems wrong to me. They then wrap it all up with:

Connected TV is TV without the clutter—the benefits of attentiveness and emotion, with better chance for ad break-through…

Yet there’s probably more clutter on Connected TV than on regular TV. You’ve not only got linear ads running but you’ve also got display ads, in app ads, etc. What this report really seems to be to me, is a vehicle to push YuMe’s connected TV platform and try to get advertisers to move more toward it. The data seems skewed or at the very least, poorly analyzed, and the participant number is extremely low. The margin of error could be in the 8% range but I suspect it might be even as high as 10%. So while this might prove interesting to read, it definitely needs a large grain of salt to go along with it. I think we, as an industry, need to sit down and talk about data reporting. I also think that, if we’re going to use terms like “arousal” or “excitement” in relation to video advertising, we need to clearly define them. We also need to clearly define ad clutter, because I just don’t see it applying to linear TV in general, yet in this study they did just that. Personally, I was unconvinced by the findings in this study, you can draw your own conclusion and find the full report over at YuMe.