Posts tagged T-Mobile
Advertising Effecitveness +OdinText

ad testing +OdinText [Authors note. As I am writing this blog post early Monday morning after the Super Bowl, I have already completed the initial ad testing analysis. It’s the case where modern AI and analytics software (OdinText) is faster than the data collection process/vendor we’re relying on. We’ve asked an open ended comment question among n=3,000 respondents about which super bowl ads they like/dislike and why. Eager to have the analysis complete as soon as possible, the analysis is already done, and blog written based on n=1,011 initial responses received. But since 1,998 more are expected I’m painfully waiting to publish results until the rest of the fielding comes in. The bad part is waiting for the sample. The good part is knowing that now repeating the analysis will literally take less than 1 minute. Just uploading the data into OdinText, and then the brand names and advertisement likes and dislikes will automatically be coded, analyzed and charted in seconds. I just have to review if anything has changed materially and make small updates in my copy below in such case. As it turned out, more data did change findings, and so I did have to change my blog copy. Ah, the joys of modern analytics!]

 

The Advertising Pundits weighed in on which ads were best and worst even before the Super Bowl aired. We tend to do things a little differently at OdinText and allow data, not opinion to drive.

Of course, for "best" and "worst" not to be subjective, we need some definition of desired outcomes. Last year we looked at a simple formula to evaluate efficacy consisting of Awareness + Positive Sentiment/Liking of the ads.

For instance, you may not remember this because of the low sentiment, but last year 85 Lumber was one of the companies with the highest Awareness after the super bowl. However, because it also had low sentiment and relevancy (as it dealt with the explosive issue of immigration/Trump's wall in a somewhat ambiguous way). It's probably the case that it ended up doing better among its core customer segments than among the general population, but since Super Bowl Ads are expensive, I argued that all things equal, a strategy with a broader target in mind, which aims to leave a positive impact among this broader group, should provide a better ROI. Looking at it another way, to have the most significant positive impact we want to maximize both awareness and sentiment almost equally.

With those assumptions and comments from over 3,000 respondents, OdinText's AI predicted which of the Super Bowl Ads were successful, and which were not. Below I've shown 10 Brands/ads, the best performing 6 and the worst 4.

OdinText Ad Ratings

10 superbowl ads rated

THE WINNERS

#1. THE NFL

https://www.youtube.com/watch?v=KUoD-gPDahw

 

In a year where there has been a lot of controversy surrounding NFL players taking a knee, and with a few of our respondents explicitly stating that they had boycotted the Super Bowl this year, it interesting to see the NFL advertising, and doing it so well. The NFL's Dirty Dancing with Manning and Beckham performed best, I believe in part because of its high relevance to the audience, but also for garnering high awareness together with very high positive sentiment/liking. In fact, only one other ad came close in sentiment.

#2 AMAZON ALEXA

https://www.youtube.com/watch?v=J6-8DQALGt4

 

That second most well-liked ad was Amazon's Alexa. Not as much because of its awareness (which was rather low in comparison), but because of its extremely high sentiment. The audience loved the various famous actors playing the voice of Alexa at least as much as they enjoyed NFL players Dirty Dancing.

#3 TIDE

https://www.youtube.com/watch?time_continue=1&v=doP7xKdGOKs

In 3rd place we have Tide. They earned the spot less so for sentiment (though viewers did like the ad). The reason Tide did so well was primarily due to the awareness it garnered. Tide had THE HIGHEST awareness of any Super Bowl Ad. However contrary to some of the Advertising Pundits opinions, it just wasn't quite as consistently well-liked by viewers as Dirty Dancing, and Alexa.

If you are in the camp who believe Awareness is everything, then Tide should have an even higher spot.

#3 DORITOS (& MOUNTAIN DEW ICE)

https://www.youtube.com/watch?time_continue=10&v=4eKYR_iL5eU

Doritos + Mountain Dew Ice was so close in our model, that I’m going to give them a tie for 3rd. Not awareness like Tide, but for balancing both positive sentiment and awareness perfectly. It's mix of awareness and liking was in the same proportions as NFL Dirty Dancing, just at a slightly smaller scale.

Obviously considering the audience and occasion, just like the NFL ad, Doritos especially is a highly relevant product, and as importantly the humorous approach with two extremely popular yet not commonly seen together stars (Namely Morgan Freeman and Peter Dinglage/Tyron from Game of Thrones) succeeded in the unique Combo messaging of Fire & Ice.

#5 BUD-LIGHT (NOT BUDWEISER)

https://www.youtube.com/watch?v=CxGUmtRLm5g

https://www.youtube.com/watch?v=lvTE_5c7buk

 

Budweiser is almost expected to do well. So, in a way, it may be surprising to see it doesn’t make it into our analysis (Beer, in general, did poorly especially Miller Ultra). It really should be so easy for Budweiser though. Here's a case where the occasion is more than just relevant, it's almost as if the brand has a historic Super Bowl halo effect. That said, their performance was less than impressive.

While the idea of stopping the Budweiser line to make water in an emergency could be touching for some, reminding consumers you have a good fun product may be a safer strategy than asking for kudos for merely being a good corporate citizen?

And that’s where Bud-Light’s Knight did better. Beer should be about fun…

#6 TOYOTA

https://www.youtube.com/watch?v=QwF3ipuNyfc

Here's a case where awareness was quite low, but the ad was still more liked than average compared to the other brands. Toyota and our #8 brand just barely made the list. While the setting was right "The Super Bowl," in the end perhaps the ‘Priest, an Imam, a Monk and a Rabbi' may have felt a bit less like a joke, and more like preaching…

THE LOSERS

#1 COCA-COLA

https://www.youtube.com/watch?v=-R-EEdvDrUU

Like Budweiser, we expect a lot from Coca-Cola when it comes to advertising. They’ve been pushing the diversity message for a few years now. It may be that pulling at heart strings is far harder to do than making people laugh. Coca-Cola had lower than average sentiment coupled with relatively low awareness. Not a winning combination.

#2 DODGE RAM

https://www.youtube.com/watch?v=SlbY1tGARUA

 

Dodge Ram did better than Coca-Cola at least, especially on awareness, but even concerning sentiment/liking.

The negative aspect of course in large part was the appropriateness of Martin Luther King's message at the beginning.

When it comes to ads like these though, I think we must assume, as was the case for 85 Lumber last year, that perhaps the brand knows what it's doing. They aren't there to please everyone (as you would hope is the goal of Pepsi and Coca-Cola), but to message their core audience with a ‘We Get You – Even if Everyone Else Doesn't'. And so, awareness wise, Ram did better than Amazon, Bud-Light, and Pepsi. But on an overall basis, they get dinged by the overall sentiment due to the some would say clumsy ‘MLK + Patriotic' messaging. Only time and sales will tell…

#3 T-MOBILE

https://www.youtube.com/watch?v=C-rumHvmqCA

T-Mobile was less well liked than you’d think, who doesn’t like babies right? Turns out people are getting tired of the “social responsibility ads” in their entertainment, at least that’s what they told us.

#4 DIET COKE (TWISTED MANGO)

https://www.youtube.com/watch?v=T6T6YrPA4aM

The Booby Prize. Ok, so here is a bad ad. In PR they used to say, any PR is good PR. But Diet Coke didn't do too well on either of our metrics. It had low awareness combined with even lower sentiment/liking. Diet Coke Mango, because, just no…

 

HALF TIME

A WORD ABOUT PEPSI

https://www.youtube.com/watch?v=2z3EUY1aXdY

https://www.youtube.com/watch?v=0gHYd67OumQ

Pepsi, what can I say.  You may be surprised that yet again, Pepsi performed poorly compared to the other brands I mentoned considering that their name was all over the Super Bowl during the Half Time Show. And yet, it may be that the real winner of Halftime is the brand of the performer, which this year was Justin Timberlake. We saw a similar pattern last year as well.

 

@TomHCAnderson

PS. See Your Data +OdinText

 

Text Analytics Picks the 10 Strongest Super Bowls Ads

New Text Analytics PollTM Shows Which Super Bowl Ads Really Performed Best Well, it’s been five days since the Super Bowl, and pretty much everyone has cranked out a “definitive” best-and-worst ad list or some sort of top 10 ranking. And frankly, I think a lot of them are based on the wrong metrics.

Without a doubt, what makes a Super Bowl ad great differs from what makes a “normal” ad great. So what exactly qualifies a Super Bowl ad as a success or failure?

We could look at purchase consideration or intent, likelihood to recommend, or any of a dozen or more other popular advertising metrics, but that’s not what Super Bowl advertising effectiveness is really about.

Word of mouth has always been a big one and nowadays that means social media buzz. But does buzz equate to success? Ask the folks at Budweiser or Lumber 84.

Bottom line: This is a very expensive reach buy, first and foremost, and it’s a branding exercise.  I’ve shelled out $5 million (plus production costs) for 30 seconds to make a lasting and largely unconscious impression on the world’s biggest television audience.

As far as I’m concerned there need only be three objectives then:

  1. I want you to remember the ad;
  2. I want you to remember it’s my ad;
  3. I want you to feel positive about it.

Whether or not my ad met all of these criteria can be answered with one single unstructured question in a Text Analytics PollTM and quickly be analyzed by NLP software like OdinText with more valid results than any multiple-choice instrument.

Why a Text Analytics PollTM ?

Using a Likert scale to assess recall or awareness will only provide an aided response; I can’t ask you about an ad or brand without mentioning it. So I don’t really know if the ad was actually that memorable. And while a quantitative instrument can tell me whether or not you liked or disliked an ad, it also won’t tell me why.

Conversely, I can get the “why” from traditional qualitative tools like focus groups or IDIs, but not only would those insights be time-consuming, labor-intensive and expensive to gather, they wouldn’t be quantified.

But if I ask you to just tell me what you remember in your own words using a comment box, I can find out which ad was truly memorable, ascertain whether or not you truly recall the brand, determine whether the ad left a positive or negative impression on you and get a much deeper understanding of why. I can achieve all of this using one open-ended question. And with text analytics software like OdinText, I can quantify these results.

Which Super Bowl Ads Did “Best”?

We asked a random, gen pop sample of n=4,535 people (statistics with a confidence interval of +/- 1.46) one simple question:

“What Super Bowl ad stood out the most to you and why?”

Author’s note: We ran this survey Sunday night and closed it Monday night. We were originally planning to post the results on Tuesday, but decided to postpone it in favor of sharing what we felt were more pressing results from a Text Analytics PollTM we had conducted around President Trump’s immigration ban.

As you can see in the table below, this one simple question told us everything we needed to know…

Top 10 Super Bowl Ads: Memorability of Ad & Brand, and Degree of Positive Sentiment

The following ads are ranked according to memorability—respondents’ unaided recall of both the ad and the brand—accompanied by positive/negative sentiment breakout (blue for positive, orange for negative) in reverse order. Author’s note: The verbatim examples included here are [sic]

#10 Pepsi

 

 

As the sponsor of the Lady Gaga halftime show, one might expect Pepsi to do very well, but Lady Gaga may have literally stolen the show from Pepsi! In fact, the halftime show was actually mentioned more often in the comment data than Pepsi, and the two were infrequently mentioned together. Meanwhile, Pepsi’s ads were relatively unmemorable and much of the awareness we saw was in the form of negative sentiment.

Author’s note: Interestingly, social media monitoring services like Sprinkler had reported Pepsi “owned” the Super Bowl ad chatter on social media. I’ll say it not for the first time: social media (aka Twitter) can be full of spam often generated by agents of the brand.

 

#9 Buick

This is a case where the star of the ad, Cam Newton, didn’t eclipse the sponsor. People liked the pro footballer playing with the little kids and the tie-in to football seemed to work well. We saw this with Tom Brady in a different ad, too.

Buick with Cam Newton, cute and funny

I like the Buick ad because it let a bunch of kids play football with Cam Newton.

So what’s not to like, you say? How did it garner even a 13% unfavorable rating?

cam newton pushed little kids

The buick commercial, the concept was boring

Buick, it was not even funny

 

#8 Skittles

 

Skittles, made my kids laugh

The Skittles ad because it was funny and sort of relate-able. It shows how far one is willing to do something for someone.

Humor generally always does well, so what’s not to like?

The skittles commercial it made no sense

skittles, stupid with the burglar

Skittles, it was creepy. And what was with the gopher at the end?

 

#7 T-Mobile

Popular and a little risqué… [Note Also, Sprint Ads were often mis-remembered as T-Mobile, perhaps Halo effect and a reason Sprint didn't make the Top 10...]

The T-Mobile ‘fake your own death to escape Verizon bill’ it was very funny, and got its point across very well

T-mobile. very funny parodying 50 shades of gray to Verizon ‘screwing its customers!’

T-Mobile with Justin. Maybe because I'm a T-Mobile subscriber? Or Justin Bieber was dressed so well in a suit, and then he starts dancing and jumping like a maniac. The contrast makes it funny.

T mobile add where guy faking death. Most memorable. Light hearted. Got point across.

BUT not everyone is a Belieber

The t mobile justin biber. It was kinda lame

T-Mobile w/Justin Bieber - inane, juvenile, bordering on insulting

T-Mobile Unlimited Moves. It wasnt funny and Justin Bieber looked like the six flags guy.

T-Mobile, awkward dancing as they attempted to appeal to teenagers

 

#6  Audi

Audi took on gender equality with an appeal to fathers of daughters. The resulting ad was memorable in 6th place:

The audi one because it was meaningful

Audi - moving story and loved the message of what to tell daughters!

Audi. I have a daughter

Audi - moving story and loved the message of what to tell daughters!

However, not everyone liked mixing politics or social issues with their football (as we will see again for some of the other top ads):

AUDI and 84 Lumber. Keep your political message out of my entertainment

Least liked Audi because it was a liberal ad

 

#5 Coca-Cola

Ironically, even without sponsoring the halftime show, Coca-Cola beat Pepsi.

The coke commercial was really meaningful and symbolic

Coca Cola because of the embracing of diversity

Coca Cola True portrayal of America's diversity

The coke ad. I liked the pro-refugee stance.

coke america is beautiful commercial, very admirable

Coca Cola Commercial because it's all about being connected

Coke , showing we are still interconnected regardless of ethnicity

I liked the coca cola ad at the very beginning. I've seen it before but I think the message is so powerful and the commercial is beautifully executed.

But the ad was not received well by many, likely in part due to the politically-charged climate. Several advertisers ran messages that struck people as being politically biased or advancing a political agenda—something not everyone cared for…

Didn’t appreciate Coca Cola trying to make a political statement

I didn't like the Coke commercial. They showed it two years ago and the year before.

Google and Coke because they shoved their political views into my face.

 

#4 Mr. Clean

Who would have predicted MR. Clean for fourth place? The brand made good use of humor, and it stood out from the other ads by targeting women (but appealed to members of both genders).

Mr clean, it was funny - Female

Mr. Clean because I'm bald -  Male

Mr. Clean, relatable, memorable, hilarious. -Female

The Mr. Clean commercial, it was funny, tasty, and got the point across. Incredibly well done ad. – Male

Mr clean because my wife pointed it out – Male

mr clean because it relates to family, and parents that stay at home and clean. it was family friendly - Female

mr clean everything else sucked – Gender Not Specified

Some men though didn’t see the humor and or get the point, calling it “weird”. It wasn’t really that they disliked it intensely; they just felt it wasn’t for them.

 

#3 Lumber 84

Not many had heard of this company before the Super Bowl, but I’ll bet you know who they are now. The third most memorable ad, yes, but more than half of those who remembered it had nothing nice to say!

First, among those who liked the ad:

It was so touching

Audi, 84 lumber, both showed compassionate ads

84 lumber - it's the only one I can remember

84 Lumber - Showed what America is actually supposed to be.

they were obviously trying to get across a non- traditional message that didn't seem to be advertising. Also it was beautifully and compellingly produced.

Lumber 84 showed that not everyone wants a wall and that we understand there is power in diversity.

But the execution confused people and whatever the intention, the sponsor stepped into a controversy. Here the emotional sentiment (particularly anger) ran high and was prevalent in comments like “romanticized crime” and “forced politics”:

The Journey 84 ad, it just left me confused

The 84 lumber commercial. It didn't make sense

it was about illegals sneaking into America, i won't be shopping their anymore

Lumber 84 because it was politically offensive

84 lumber, clearly a political statement and uncalled for

84 lumber, Made no sense, Not going to look something up

#2 Kia

Ironically, with other brands going serious and political, Kia poked some fun with help from Melissa McCarthy. Kia’s investment in humor and McCarthy paid off in a big way, scoring the highest combination of memorability and positive sentiment, although to an extent the comedian eclipsed the brand.

Loved melissa McCarthy because she is hilarious and i love her.

Kia it was funny and not somber like most the others

The Buick one, the world of tanks ones and the eco friendly Melissa one because they were the funniest

The one with Melissa McCarthy because it made me laugh

KIA becuase it didn't feel like it was trying to sell me anything, just entertain with brand placement

 

#1 Budweiser

Yes, Budweiser took first place in terms of recall, but the perception of a political bent cost the king of beers. The ad, which featured one of the founders struggling as an immigrant, was apparently in the works before the Trump Immigration Order controversy. But even if that was the case, by choosing to air it Budweiser took a risk.

Likes:

I liked the Budweiser commercial reminded us all that not all white Europeans were always welcome in the US.

Budweiser. I love the reminder that we are all immigrants

Budweiser immigration. Shows Trump is an idiot, but we all know that

The Budweiser ad about how they were founded by an immigrant, because it was actually relevant to their company history

Budweiser, it was a beautiful immigrant's tale. Not overtly political

The Budweiser commercial because it shows what a true immigrant had to go through and even though many people thought it was to take a shot at Trump's travel ban it had nothing to do with it.

Dislikes:

Budweiser. Too liberal.

budweiser, too pro immigration

bud, adolfus was not ILLEGAL !

The Budweiser ad about immigration. Too political.

Budweiser, they shot themselves in the foot being that the man who immigrated into the U.S. did so legally.

Budweiser. Football/all sports should not involve politics. We need to relax sometimes.

So…who won?

Isn’t it obvious? I’d say Kia. Sure, Budweiser scored higher unaided awareness, but a significant portion of that was negative.

But it's all in the data, what do you think?

A Final Note on Text Analytics PollsTM 

It occurred to me in writing this post that about 11 years ago almost to the day I predicted that the survey of the future would be a one-question open-end, because that’s all people really want to tell you, and that’s all you’ll need.

Turns out I may have been right.

This week, we’ve shared results from three such surveys, a technique we've dubbed “Text Analytics PollTM .

These incredibly short, one-question polls allow us to field quickly to large samples with minimal burden on the respondent. And text analysis software such as OdinText enables us to quantify these huge quantities of comments.

But the real advantage to using text analytics polls is that the responses tell us so much more than whether someone agrees/disagrees or likes/dislikes. Using text analytics we can uncover why from respondents in their own words.

Thanks again for reading!

@TomHCAnderson @OdinText

Could a text analytics poll answer your burning marketing questions?  Contact us to see if a single-question open-ended survey makes sense for you!

 

About Tom H. C. Anderson

Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the "Four under 40" market research leaders by the American Marketing Association in 2010. He  tweets under the handle @tomhcanderson.