Posts in Emotion Analysis
Of Tears and Text Analytics

An OdinText User Story - Text Analytics Tips Guest Post (AI Meets VOC)

Today on the blog we have another first in a soon to be ongoing series. We’re inviting OdinText users to participate more on the Text Analytics Tips blog. Today we have Kelsy Saulsbury guest blogging. Kelsy is a relatively new user of OdinText though she’s jumped right in and is doing some very interesting work.

In her post she ponders the apropos topic, whether automation via artificial intelligence may make some tasks too easy, and what if anything might be lost by not having to read every customer comment verbatim.

 

Of Tears and Text Analytics By Kelsy Saulsbury Manager, Consumer Insights & Analytics

“Are you ok?” the woman sitting next to me on the plane asked.  “Yes, I’m fine,” I answered while wiping the tears from my eyes with my fingers.  “I’m just working,” I said.  She looked at me quizzically and went back to reading her book.

I had just spent the past eight hours in two airports and on two long flights, which might make anyone cry.  Yet the real reason for my tears was that I had been reading hundreds of open-end comments about why customers had decided to buy less from us or stop buying from us altogether.  Granted eight hours hand-coding open ends wasn’t the most accurate way to quantify the comments, but it did allow me to feel our customers’ pain from the death of a spouse to financial hardship with a lost job.  Other reasons for buying less food weren’t quite as sad — children off to college or eating out more after retirement and a lifetime of cooking.

I could also hear the frustration in their voices on the occasions when we let them down.  We failed to deliver when we said we would, leaving the dessert missing from a party.  They took off work to meet us, and we never showed.  Anger at time wasted.

Reading their stories allowed me to feel their pain and better share it with our marketing and operations teams.  However, I couldn’t accurately quantify the issues or easily tie them to other questions in the attrition study.  So this year when our attrition study came around, I utilized a text analytics tool (OdinText) for the text analysis of our open ends around why customers were buying less.

It took 1/10th of the time to see more accurately how many people talked about each issue.  It allowed me to better see how the issues clustered together and how they differed based on levels of overall satisfaction.  It was fast, relatively easy to do, and directly tied to other questions in our study.

I’ve seen the benefits of automation, yet I’m left wondering how we best take advantage of text analytics tools without losing the power of the emotion in the words behind the data.  I missed hearing and internalizing the pain in their voices.  I missed the tears and the urgency they created to improve our customers’ experience.

 

Kelsy Saulsbury Manager, Consumer Insights & Analytics Schwan's Company

 

A big thanks to Kelsy for sharing her thoughts on OdinText's Text Analytics Tips blog. We welcome your thoughts and questions in comment section below.

If you’re an OdinText user and have a story to share please reach out. In the near future we’ll be sharing more user blog posts and case studies.

@OdinText

How Fear of Frexit Helped Macron Win the French Presidential Election
NEW Text Analytics PollTM Shows a Trump-Style Le Pen Upset May Have Been Averted by Overwhelming Opposition to a Frexit

Last week on this blog, I reported findings from a Text Analytics Poll™ of 3,000 French citizens showing that Marine Le Pen’s positioning going into the runoff looked remarkably similar to that of another recent underdog candidate, Donald Trump, just days before his stunning U.S. election upset.

Indeed, a similar set of circumstances appeared to be in play, as noted by the New York Times in an article on Election Day: “Populist anger at the political establishment; economic insecurity among middle class voters; public alienation toward mainstream political parties; rising resentment toward immigrants.”

Yet on Sunday, the French people elected Emmanuel Macron president over Le Pen by about 66/34. So why wasn’t the race closer?

The answer may be in data we collected from French and British respondents, which shows that the prospect of a Le Pen “Frexit” probably figured highly in Macron’s victory.

Positioning: Voting Against a Candidate

Our data in the French presidential poll were eerily reminiscent of data we collected prior to the U.S. election, which suggested a victory may not so much amount to an endorsement of one candidate as a rejection of the other.

Our analysis showed that first and foremost, the French associated Le Pen with bigotry and hatemongering, but text analysis also showed that among the French she was strongly positioned around immigration reform and putting France first—a platform that worked effectively for Trump, who had also been labeled a bigot in the minds of many Americans. In fact, the perception of Trump as a bigot was only slightly lower among Americans than the perception of Le Pen as a bigot among the French (11% vs 15%, respectively).

In contrast, respondents most frequently associated Macron with “liberalism”—meaning economic liberalism favoring free markets—followed by capitalism, neither of which is necessarily an asset in terms of positioning in French politics, particularly for a wealthy investment banker at a time when job security is a major concern among middleclass voters.

But the main platform issue that people associated with Macron—which trailed just behind people’s view of him as a proponent of free markets/capitalism—was Europe/EU, in stark contrast to Le Pen, who was well known to strongly favor an EU “Frexit.” The EU is also synonymous with the free movement of commerce and people, which, of course, stands in contrast to the dual protectionist/anti-immigration platform championed by Le Pen.

This, naturally, begged the question: How important is EU membership to the French population?

If the mood of the French electorate were anything like that of British Brexit voters, then favoring EU membership could be a liability. So just days ahead of the election we ran a second Text Analytics Poll—once again a single question—only this time we polled 3000 voters each in France and the UK:

  1. “What does the European Union mean to you?” (or “Qu'est ce que l'Union Européenne représente pour vous?” in French).

EU Membership Means “Hope”

It’s worth noting that turnout for this election was reportedly the lowest in 36 years. These were presumably voters who never would’ve cast a ballot for Le Pen, but who also could not be mobilized for Macron. In short, they were Macron’s to lose.

This new poll data helps explain why, in spite of inspiring lackluster confidence and support from anti-Le Pen voters, Macron nonetheless won the election by a sizable margin.

EU UK V FRANCE

While a significant number of the French tell us the EU means nothing to them, this is significantly lower than the Brits who say so.

Conversely, the French are more than five times as likely as Brits to say the EU means “Everything/A Lot” to them. The French are also far less likely than their UK counterparts to criticize the EU for corruption, wastefulness and such.

Instead, the French are extremely optimistic about the EU, with many indicating it provides “future hope” and keeps them out of wars and at “Peace” —something Brits are more likely to attribute to NATO.

High Positive Emotions for EU

Ultimately, emotions are what really drive behavior, and in the end, the French electorate’s highly positive emotional disposition toward the EU—notably their “Anticipation” and hopefulness—may have countered Macron’s relatively weak positioning in this election.

eMOTIONS TOWARD EU 2

Closing Thoughts

I read some responses to our original analysis that I’d characterize as emotionally overwrought. I understand that this is an occupational hazard for anyone conducting political opinion research, but our duty is to present and report objectively what the data tells us—even if what we’re seeing in the data isn’t necessarily pleasant.

The job of these polls was to assess the candidates’ brand positioning in the minds of voters, and to review the potential opportunities and threats in the “marketplace” as we would for any brand.

I want to stress that I am not discounting people’s distaste for Marine Le Pen’s perceived bigotry as being a key factor behind her loss in this election, but I’ll emphasize again that it was only slightly higher (15% vs 11%) than what we saw for Donald Trump, who, as you know, is now the President of the United States.

And at the end of the day, the hard truth is that more than a third of those who voted in this election voted for a right-wing nationalist—a candidate whose background makes Donald Trump look like a civil rights activist by comparison. Moreover, 25% of the electorate were not sufficiently affronted by Madame Le Pen’s politics to at least vote against her by voting for Macron; instead, they just abstained.

Like many people, I am relieved by the outcome of this election, but it seems clear from the positioning of both candidates—as reported by French citizens, unaided, in their own words—and the data on EU membership from our second poll that the French people did not simply reject Marine Le Pen because she is positioned as a racist/hatemonger; she was on the wrong side of Frexit.

@TomHCAnderson

*Note: n=3,000 responses were collected via Google Surveys 3/3-5/5 2017. Google Surveys allow researchers to reach a validated French General Population Representative sample by intercepting people attempting to access high-quality online content or who have downloaded the Google Opinion Rewards mobile app. Results are +/- 2.51% accurate at the 95% confidence interval.

Text Analytics Tips

About Tom H. C. Anderson Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the “Four under 40” market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson.

Text Analytics Reveals Potential French Election Upset
Text Analytics Poll Shows Le Pen Positioned to “Trump” Macron

To Americans following the French Presidential Election taking place in less than a week, it might appear as though recent history is repeating itself. And in many ways, it is.

Late last week we ran a Text Analytics Poll™ in France, and the results of our analysis bear a striking resemblance to those of an identical poll we ran in the US just a couple of days prior to the November 8 presidential election.

You may recall that in November, just a day before the US Presidential Election, we posted on this blog results from a Text Analytics Poll™ indicating that Hillary Clinton had a major positioning problem that could cost her the election, in contrast to conventional pollsters’ predictions that had almost universally and, it turns out, incorrectly forecast her winning by a sizable Electoral College margin.

Well, as was the case in our US poll, actual comment data from French respondents in their own words indicates a much, much closer race between Emmanuel Macron and Marine Le Pen than the 60/40 split pollsters have thus far predicted.

In fact, as of Sunday night when we closed this poll—exactly one week before the May 7 runoff election—Marine Le Pen looked a lot like Donald Trump.

About this Text Analytics Poll™

For this French election Text Analytics Poll™, we replicated our November US election poll, taking a French general population sample of 3,000, splitting it in half randomly, and asking each half the same single question, substituting only the candidate’s name:

“Without looking, off the top of your mind, what issues does [insert candidate name] stand for?”

We then machine-translated the responses and analyzed them using the patented OdinText software platform, which identified and quantified potentially important themes/ideas/topics in people’s comments and also qualified and quantified the emotions expressed in those comments.

We use this approach because we’ve found time and again that conventional quantitative survey questions—the sort used in political polls—are usually not terrific predictors of actual behavior.

We know that consumers (and, yes, voters) are generally not rational decision-makers; people rely on emotions and heuristics to make most of our decisions. Ergo, if I really want to understand what will drive actual behavior, the surest way to find out is by allowing you to tell me unaided, in your own words, off the top of your head. Oftentimes, we can accomplish this with one, well-designed question!

French Election Outsider vs. Reformer

Much as we saw in the US race, the French electorate appears to be in a decidedly anti-establishment mood. So it’s no surprise under the circumstances that both of the final contenders in the French presidential runoff could accurately be described as “outsiders,” but what voters may really be after is a reformer.

Bernie Sanders and Donald Trump were both considered outsiders and reformers, although unlike Trump, who successfully hijacked the Republican nomination, Sanders failed to pull off a similar grassroots coup in the Democratic primary. As a result, US voters were faced with a choice between a reformer/outsider and an establishment candidate.

Le Pen has been a member of the French Parliament for more than a decade and she held elected office at the regional level before that. She’s also the scion of a famous political family and, more importantly, the former president of a prominent, albeit right wing, political party, the National Front (FN). Le Pen’s relative “outsider” status stems from the fact that the FN has historically promoted a nationalist agenda and was until recently viewed as outside of the political mainstream (and outside the two major coalitions that have alternated between control of the French government for the last 30-plus years).

Emmanuel Macron, too, is a relative outsider. He’s a former Minister of the Economy and founded the “En Marche”(“Forward!”) political movement in 2016, but he has never held elected office and, as of our poll, remains something of a mystery to potential French voters save for the fact that it’s well known that he made a fortune in investment banking.

Whatever you think of her politics, Le Pen clearly qualifies as a reformer, whereas Macron, while an outsider, appears to have a positioning problem around reform. Let’s take a closer look…

It’s All About Brand Positioning… Again

Whether you’re a corporation or a candidate for office, properly positioning your brand in the mind of your target is arguably the single most important part of the marketing process.

As I noted, our US poll back in November strongly suggested that Hillary Clinton was in more trouble than any of the other polling data to that point had indicated, and the problem was one of positioning relative to the competition.

Why?

- The #1 most popular response for Hillary Clinton involved the perception of dishonesty/corruption.

- The #1 and #2 most popular responses for Donald Trump related to platform (immigration, followed by pro-USA/America First), followed thirdly by perceived racism/hatemongering.

Again, I’ll emphasize that these responses were not selected from a list of possible choices, but top-of-mind and unaided from voters in their own words.

What the comment data revealed was that Donald Trump’s campaign messaging was very focused around a two issues—immigration and protectionism—and had been effective in galvanizing voters to whom these positions appealed; Hillary Clinton’s messaging was relatively scattered across a variety of issues, and therefor diluted, which made it difficult for voters to identify her with a key issue they could rally around.

And while an alarmingly high proportion of responses to our question were for both candidates emotionally-charged character attacks, the negative emotional disposition toward Hillary Clinton was actually higher than for Donald Trump. In other words, the dislike among people who disliked Hillary Clinton outweighed the dislike among people who disliked Donald Trump. This probably had little to do with Trump campaign messaging—although they certainly capitalized on it—and was more a reflection of the fact that Hillary Clinton had been highly visible and active in national politics for decades and was already positioned in the minds of voters.

How does this relate to what we see in the French Election data?

The chart below depicts responses from the French to our single question after being analyzed by OdinText and sorted by prevalence of topics/themes (coded red for Macron and blue for Le Pen).

First, it’s important to note that there are inherently fewer issues with which politicians can differentiate themselves in French politics than there are in US politics. For example, issues like abortion, education, healthcare, gun ownership, etc., in France are not hotly contested as they are in the States.

In France—like most European countries in the post-Brexit era—political debate centers primarily around economics internally and in relation to other countries (i.e. the EU), security, and, importantly, immigration.

Here, Le Pen’s positioning is unmistakable, as she was frequently associated with immigration, which works in her favor among those who view immigration as a problem. The issue is tied to security, as well, and given the 2015 Paris attacks, the heightened fear about terrorism coupled with domestic economic concerns could lead voters who might have been historically more sympathetic to pro-immigration platforms to actually vote for Le Pen.

That said, like Hillary Clinton, Marine Le Pen is well known to the French, and already positioned in their eyes. Although she has taken steps to soften the perception, respondents to our poll most frequently said she stands for racism/hate/xenophobia, which does not bode well for her candidacy in socially liberal France.

Macron, by contrast, remains a relative enigma to the French people. Almost twice as many French people said they aren’t sure what Macron stands for compared to Le Pen. In fact, Macron is not tied to any standout platform or issue of importance to the French, whereas Le Pen is positioned as a reformer on immigration to an electorate that, again, is not enamored with the status quo.

Moreover, respondents most frequently associated Macron with “liberalism,” followed by capitalism, which are nearly the same. Indeed, I put liberalism in quotes here to make a very important distinction that might have otherwise been lost on Americans who are not familiar with French politics: Liberalism in France actually refers to economic liberalism favoring free markets—almost the opposite of how the term is used in US politics!

Neither liberalism nor capitalism are necessarily assets in terms of positioning in French politics, particularly for a wealthy investment banker at a time when job security is a major concern among voters. Macron has campaigned as a centrist, stating emphatically that ideologically he is neither left nor right, but our data suggests that he is positioned in the minds of the French as something of a neo-conservative and perhaps an elite. Indeed, the Le Pen campaign has been feeding this positioning and tying it to fears about globalization undermining the economic security of the French people.

We do see in the data that Le Pen’s positioning of Macron as a capitalist “sell-out” and instrument of status quo globalists has achieved some success, but it may be too little too late. While 7.8% of the French in our poll view Macron as capitalist/money man, nearly twice as many describe Le Pen as a hatemongering racist (15.3%).

Ironically, we noted in our US poll that Donald Trump was also described as a racist by more than 10% of Americans just days before the election; however, more than 12% of Americans said that Hillary Clinton was dishonest/“crooked.”

The combined chart below shows how both the French and the American candidates appeared in the eyes of respondents from their respective countries. (Again, note that “liberal” for Macron does not mean fiscal or socially liberal as it does in the context of US politics, but refers to free-market economic liberalism.)

French Election 4

Final Analysis

This upcoming election is actually runoff, and the opponents have basically two weeks to position one another. To this point, the job of defining one’s opponent was much trickier because there were five candidates in the race. In US politics, obviously, candidates have a lot more time to cement positioning against a single opponent.

But French campaign strategists are accustomed to operating within this short timeline. The Macron campaign has enjoyed an advantage in that negative positioning around Le Pen was already firmly in place, whereas Macron was relatively unknown. Conversely, the Le Pen campaign now has a huge opportunity to negatively position Macron as an instrument of global bankers and the status quo and to sway voters with a message of protectionism and security at a time when both have high appeal.

The wild card here is the EU. An EU “Frexit” is generally accepted to be less appealing among the majority of French, and although Le Pen has been softening her rhetoric, she is known to strongly favor leaving the EU. Macron, however, is most assuredly opposed to a Frexit, and the data show that respondents understand this difference.

Much like we saw in the US election results foreshadowed by our own polling data, a victory in this election may not so much amount to an endorsement of one candidate as a rejection of the status quo. And of the two candidates, Le Pen is better positioned as the reformer. She could yet ride a wave of populism that Macron is not equipped to tap into.

In short, do not be surprised if Marine Le Pen pulls off a Trump-style upset in the French Presidential Election. The data strongly suggest she is positioned to do so!

@TomHCAnderson

*Note: n=3,000 responses were collected via Google Surveys 4/24-4/30 2017. Google Surveys allow researchers to reach a validated French General Population Representative sample by intercepting people attempting to access high-quality online content or who have downloaded the Google Opinion Rewards mobile app. Results are +/- 2.51% accurate at the 95% confidence interval.

Text Analytics Tips

About Tom H. C. Anderson Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the "Four under 40" market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson.

Text Analytics: It's Not Just for BIG Data

In a world focused on the value of Big Data, it's important to realize that Small Data is meaningful, too, and worth analyzing to gain understanding. Let me show you with a personal example. If you're a regular reader of the OdinText blog, you probably know that our company President, Tom Anderson, writes about performing text analytics on large data sets.  And yes, OdinText is ideal for understanding data after launching a rapid survey then collecting thousands of responses.

However for this blog post, I'm going to focus on the use of Text Analytics for smaller, nontraditional data set:  emails.

SMALL Data (from email) Text Analytics

I recently joined OdinText as Vice President, working closely with Tom on all our corporate initiatives. I live in a small town in Connecticut with an approximate population of 60,000.  Last year I was elected to serve our town government as an RTM member along with 40 other individuals.  Presently, our town's budget is $290M and the RTM is designing the budget for the next year.

Many citizens email elected members to let them know how they feel about the budget.  To date, I have received 280 emails. (Before you go down a different path with this, please know that I respond personally to each one -- people who take the time to write me deserve a personal response.  I did not and will not include in this blog post how I intend to vote on the upcoming budget, nor will I include anything about party affiliations. And I certainly will not share names.)

As the emails were coming in, I started to wonder … what if I ran this the data I was receiving through OdinText?  Would I be able to use the tool to identify, understand and quantify the themes in the people’s thoughts on how I should vote on the budget?

The Resulting Themes from Small Data Analytics

A note about the methodology:  Each email that I received contained the citizen's name, their email address and content in open text format.  Without a key driver metric like OSAT, CSAT or NPS to analyze the text against, I chose to use overall sentiment. Here is what I learned

Emails about the town budget show that our citizens feel Joy but RTM members need to recognize their Sadness, Fear and Anger

Joy:

“I have been a homeowner in Fairfield for 37 years, raised 4 kids here and love the community.”

Sadness:

“I am writing you to tell you that I am so unhappy with the way you have managed our town.”

Fear:

“My greatest concern seems to be the inability of our elected members to cut spending and run the town like a business”

Anger:

“We live in a very small house and still have to pay an absurd amount of money in taxes.”

Understanding the resulting themes in their own words

Reduce Taxes (90.16%)

“Fairfield taxes are much higher than surrounding communities.”

“Fairfield taxes are out of line with similar communities”

“The town has to stop raising taxes at such a feverish rate.”

“High taxes are slowly eroding the town of Fairfield.”

Moving if Taxes are Increased (25.13%)

“I am on a fixed income at 64, and cannot afford Fairfield’s taxes now. Please recognize that I cannot easily sell my house, due to the economy & the amount of homes on the market here”

“regret to say most of our colleagues and friends have an "exit strategy" to leave Fairfield”

“Our town is losing residents who are fed up and have moved or are moving to Westport and other towns with lower mil rates”

Reduce Spending (33.33%)

“... bring spending under control”

“Stop the spending please”

“... needs to trim fat at the local level, cut services, stop spending money”

“We need to keep taxes down as much as possible - even if it means spending cuts.”

Education ‘don’t cut’ (8.74%)

“… takes great pride in its education system”

“… promise of an excellent public education”

“… fiscal responsibility; however, not at the expense of the children and their right to an excellent education.”

Education ‘please cut’ (9.83%)

“Let's shave funding from all programs including education”

“... deeply questioning our education budget”

“... reduce the Education budget”

“I have a cherished budgetary item that I want protected--the library. Cut that last, after you cut education, police, official salaries”

Big Value from Small Data in Little Time

I performed this text analysis in 30 minutes. Ironically, it has taken me longer to write this blog post than it did to quantify the text from all those emails. Yet the information and understanding I have gleaned will empower me as I make decisions on this important topic. A small investment in small data has paid off in a BIG way.

Tim Lynch - @OdinText

Text Analytics Picks the 10 Strongest Super Bowls Ads

New Text Analytics PollTM Shows Which Super Bowl Ads Really Performed Best Well, it’s been five days since the Super Bowl, and pretty much everyone has cranked out a “definitive” best-and-worst ad list or some sort of top 10 ranking. And frankly, I think a lot of them are based on the wrong metrics.

Without a doubt, what makes a Super Bowl ad great differs from what makes a “normal” ad great. So what exactly qualifies a Super Bowl ad as a success or failure?

We could look at purchase consideration or intent, likelihood to recommend, or any of a dozen or more other popular advertising metrics, but that’s not what Super Bowl advertising effectiveness is really about.

Word of mouth has always been a big one and nowadays that means social media buzz. But does buzz equate to success? Ask the folks at Budweiser or Lumber 84.

Bottom line: This is a very expensive reach buy, first and foremost, and it’s a branding exercise.  I’ve shelled out $5 million (plus production costs) for 30 seconds to make a lasting and largely unconscious impression on the world’s biggest television audience.

As far as I’m concerned there need only be three objectives then:

  1. I want you to remember the ad;
  2. I want you to remember it’s my ad;
  3. I want you to feel positive about it.

Whether or not my ad met all of these criteria can be answered with one single unstructured question in a Text Analytics PollTM and quickly be analyzed by NLP software like OdinText with more valid results than any multiple-choice instrument.

Why a Text Analytics PollTM ?

Using a Likert scale to assess recall or awareness will only provide an aided response; I can’t ask you about an ad or brand without mentioning it. So I don’t really know if the ad was actually that memorable. And while a quantitative instrument can tell me whether or not you liked or disliked an ad, it also won’t tell me why.

Conversely, I can get the “why” from traditional qualitative tools like focus groups or IDIs, but not only would those insights be time-consuming, labor-intensive and expensive to gather, they wouldn’t be quantified.

But if I ask you to just tell me what you remember in your own words using a comment box, I can find out which ad was truly memorable, ascertain whether or not you truly recall the brand, determine whether the ad left a positive or negative impression on you and get a much deeper understanding of why. I can achieve all of this using one open-ended question. And with text analytics software like OdinText, I can quantify these results.

Which Super Bowl Ads Did “Best”?

We asked a random, gen pop sample of n=4,535 people (statistics with a confidence interval of +/- 1.46) one simple question:

“What Super Bowl ad stood out the most to you and why?”

Author’s note: We ran this survey Sunday night and closed it Monday night. We were originally planning to post the results on Tuesday, but decided to postpone it in favor of sharing what we felt were more pressing results from a Text Analytics PollTM we had conducted around President Trump’s immigration ban.

As you can see in the table below, this one simple question told us everything we needed to know…

Top 10 Super Bowl Ads: Memorability of Ad & Brand, and Degree of Positive Sentiment

The following ads are ranked according to memorability—respondents’ unaided recall of both the ad and the brand—accompanied by positive/negative sentiment breakout (blue for positive, orange for negative) in reverse order. Author’s note: The verbatim examples included here are [sic]

#10 Pepsi

 

 

As the sponsor of the Lady Gaga halftime show, one might expect Pepsi to do very well, but Lady Gaga may have literally stolen the show from Pepsi! In fact, the halftime show was actually mentioned more often in the comment data than Pepsi, and the two were infrequently mentioned together. Meanwhile, Pepsi’s ads were relatively unmemorable and much of the awareness we saw was in the form of negative sentiment.

Author’s note: Interestingly, social media monitoring services like Sprinkler had reported Pepsi “owned” the Super Bowl ad chatter on social media. I’ll say it not for the first time: social media (aka Twitter) can be full of spam often generated by agents of the brand.

 

#9 Buick

This is a case where the star of the ad, Cam Newton, didn’t eclipse the sponsor. People liked the pro footballer playing with the little kids and the tie-in to football seemed to work well. We saw this with Tom Brady in a different ad, too.

Buick with Cam Newton, cute and funny

I like the Buick ad because it let a bunch of kids play football with Cam Newton.

So what’s not to like, you say? How did it garner even a 13% unfavorable rating?

cam newton pushed little kids

The buick commercial, the concept was boring

Buick, it was not even funny

 

#8 Skittles

 

Skittles, made my kids laugh

The Skittles ad because it was funny and sort of relate-able. It shows how far one is willing to do something for someone.

Humor generally always does well, so what’s not to like?

The skittles commercial it made no sense

skittles, stupid with the burglar

Skittles, it was creepy. And what was with the gopher at the end?

 

#7 T-Mobile

Popular and a little risqué… [Note Also, Sprint Ads were often mis-remembered as T-Mobile, perhaps Halo effect and a reason Sprint didn't make the Top 10...]

The T-Mobile ‘fake your own death to escape Verizon bill’ it was very funny, and got its point across very well

T-mobile. very funny parodying 50 shades of gray to Verizon ‘screwing its customers!’

T-Mobile with Justin. Maybe because I'm a T-Mobile subscriber? Or Justin Bieber was dressed so well in a suit, and then he starts dancing and jumping like a maniac. The contrast makes it funny.

T mobile add where guy faking death. Most memorable. Light hearted. Got point across.

BUT not everyone is a Belieber

The t mobile justin biber. It was kinda lame

T-Mobile w/Justin Bieber - inane, juvenile, bordering on insulting

T-Mobile Unlimited Moves. It wasnt funny and Justin Bieber looked like the six flags guy.

T-Mobile, awkward dancing as they attempted to appeal to teenagers

 

#6  Audi

Audi took on gender equality with an appeal to fathers of daughters. The resulting ad was memorable in 6th place:

The audi one because it was meaningful

Audi - moving story and loved the message of what to tell daughters!

Audi. I have a daughter

Audi - moving story and loved the message of what to tell daughters!

However, not everyone liked mixing politics or social issues with their football (as we will see again for some of the other top ads):

AUDI and 84 Lumber. Keep your political message out of my entertainment

Least liked Audi because it was a liberal ad

 

#5 Coca-Cola

Ironically, even without sponsoring the halftime show, Coca-Cola beat Pepsi.

The coke commercial was really meaningful and symbolic

Coca Cola because of the embracing of diversity

Coca Cola True portrayal of America's diversity

The coke ad. I liked the pro-refugee stance.

coke america is beautiful commercial, very admirable

Coca Cola Commercial because it's all about being connected

Coke , showing we are still interconnected regardless of ethnicity

I liked the coca cola ad at the very beginning. I've seen it before but I think the message is so powerful and the commercial is beautifully executed.

But the ad was not received well by many, likely in part due to the politically-charged climate. Several advertisers ran messages that struck people as being politically biased or advancing a political agenda—something not everyone cared for…

Didn’t appreciate Coca Cola trying to make a political statement

I didn't like the Coke commercial. They showed it two years ago and the year before.

Google and Coke because they shoved their political views into my face.

 

#4 Mr. Clean

Who would have predicted MR. Clean for fourth place? The brand made good use of humor, and it stood out from the other ads by targeting women (but appealed to members of both genders).

Mr clean, it was funny - Female

Mr. Clean because I'm bald -  Male

Mr. Clean, relatable, memorable, hilarious. -Female

The Mr. Clean commercial, it was funny, tasty, and got the point across. Incredibly well done ad. – Male

Mr clean because my wife pointed it out – Male

mr clean because it relates to family, and parents that stay at home and clean. it was family friendly - Female

mr clean everything else sucked – Gender Not Specified

Some men though didn’t see the humor and or get the point, calling it “weird”. It wasn’t really that they disliked it intensely; they just felt it wasn’t for them.

 

#3 Lumber 84

Not many had heard of this company before the Super Bowl, but I’ll bet you know who they are now. The third most memorable ad, yes, but more than half of those who remembered it had nothing nice to say!

First, among those who liked the ad:

It was so touching

Audi, 84 lumber, both showed compassionate ads

84 lumber - it's the only one I can remember

84 Lumber - Showed what America is actually supposed to be.

they were obviously trying to get across a non- traditional message that didn't seem to be advertising. Also it was beautifully and compellingly produced.

Lumber 84 showed that not everyone wants a wall and that we understand there is power in diversity.

But the execution confused people and whatever the intention, the sponsor stepped into a controversy. Here the emotional sentiment (particularly anger) ran high and was prevalent in comments like “romanticized crime” and “forced politics”:

The Journey 84 ad, it just left me confused

The 84 lumber commercial. It didn't make sense

it was about illegals sneaking into America, i won't be shopping their anymore

Lumber 84 because it was politically offensive

84 lumber, clearly a political statement and uncalled for

84 lumber, Made no sense, Not going to look something up

#2 Kia

Ironically, with other brands going serious and political, Kia poked some fun with help from Melissa McCarthy. Kia’s investment in humor and McCarthy paid off in a big way, scoring the highest combination of memorability and positive sentiment, although to an extent the comedian eclipsed the brand.

Loved melissa McCarthy because she is hilarious and i love her.

Kia it was funny and not somber like most the others

The Buick one, the world of tanks ones and the eco friendly Melissa one because they were the funniest

The one with Melissa McCarthy because it made me laugh

KIA becuase it didn't feel like it was trying to sell me anything, just entertain with brand placement

 

#1 Budweiser

Yes, Budweiser took first place in terms of recall, but the perception of a political bent cost the king of beers. The ad, which featured one of the founders struggling as an immigrant, was apparently in the works before the Trump Immigration Order controversy. But even if that was the case, by choosing to air it Budweiser took a risk.

Likes:

I liked the Budweiser commercial reminded us all that not all white Europeans were always welcome in the US.

Budweiser. I love the reminder that we are all immigrants

Budweiser immigration. Shows Trump is an idiot, but we all know that

The Budweiser ad about how they were founded by an immigrant, because it was actually relevant to their company history

Budweiser, it was a beautiful immigrant's tale. Not overtly political

The Budweiser commercial because it shows what a true immigrant had to go through and even though many people thought it was to take a shot at Trump's travel ban it had nothing to do with it.

Dislikes:

Budweiser. Too liberal.

budweiser, too pro immigration

bud, adolfus was not ILLEGAL !

The Budweiser ad about immigration. Too political.

Budweiser, they shot themselves in the foot being that the man who immigrated into the U.S. did so legally.

Budweiser. Football/all sports should not involve politics. We need to relax sometimes.

So…who won?

Isn’t it obvious? I’d say Kia. Sure, Budweiser scored higher unaided awareness, but a significant portion of that was negative.

But it's all in the data, what do you think?

A Final Note on Text Analytics PollsTM 

It occurred to me in writing this post that about 11 years ago almost to the day I predicted that the survey of the future would be a one-question open-end, because that’s all people really want to tell you, and that’s all you’ll need.

Turns out I may have been right.

This week, we’ve shared results from three such surveys, a technique we've dubbed “Text Analytics PollTM .

These incredibly short, one-question polls allow us to field quickly to large samples with minimal burden on the respondent. And text analysis software such as OdinText enables us to quantify these huge quantities of comments.

But the real advantage to using text analytics polls is that the responses tell us so much more than whether someone agrees/disagrees or likes/dislikes. Using text analytics we can uncover why from respondents in their own words.

Thanks again for reading!

@TomHCAnderson @OdinText

Could a text analytics poll answer your burning marketing questions?  Contact us to see if a single-question open-ended survey makes sense for you!

 

About Tom H. C. Anderson

Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the "Four under 40" market research leaders by the American Marketing Association in 2010. He  tweets under the handle @tomhcanderson.

 

 

 

Poll: What Other Countries Think of Trump’s Immigration Order

Text Analytics PollTM Shows Australians, Brits, and Canadians  Angry About Executive Order Temporarily Barring Refugee (Part II of II)In my previous post, we compared text analysis of results from an open-ended survey instrument with a conventional Likert-scale rating poll to assess where 3,000 Americans really stand on President Trump’s controversial executive order temporarily barring refugees and people from seven predominately-Muslim countries from entering the U.S.

Today, we’re going to share results from an identical international study that asked approx. 9,000 people—3,000 people from each of three other countries—what they think about the U.S. immigration moratorium ordered by President Trump.

But first, a quick recap…

As I noted in the previous post, polling on this issue has been pretty consistent insomuch as Americans are closely divided in support/opposition, but the majority position flips depending on the poll. Consequently, the accuracy of polling has again been called into question by pundits on both sides of the issue.

By fielding the same question first in a multiple-choice response format and a second time providing only a text comment box for responses, and then comparing results, we were able to not only replicate the results of the former but gain a much deeper understanding of where Americans really stand on this issue.

Text analysis confirmed a much divided America with those opposing the ban just slightly outnumbering (<3%) those who support the order (42% vs 39%). Almost 20% of respondents had no opinion or were ambivalent on this issue.

Bear in mind that text analysis software such as OdinText enables us to process and quantify huge quantities of comments (in this case, more than 1500 replies from respondents using their own words) in order to arrive at the same percentages that one would get from a conventional multiple-choice survey.

But the real advantage to using an open-ended response format (versus a multiple-choice) to gauge opinion on an issue like this is that the responses also tell us so much more than whether someone agrees/disagrees or likes/dislikes. Using text analytics we uncovered people’s reasoning, the extent to which they are emotionally invested in the issue, and why.

Today we will be looking a little further into this topic with data from three additional countries: Australia, Canada and the UK.

A note about multi-lingual text analysis and the countries selected for this project…

Different software platforms handle different languages with various degrees of proficiency. OdinText analyzes most European languages quite well; however, analysis of Dutch, German, Spanish or Swedish text requires proficiency in said language by the analyst. (Of course, translated results, including and especially machine-translated results, work very well with text analytics.)

Not inconspicuously, each of the countries represented in our analysis here has an English-speaking population. But this was not the primary reason that we chose them; each of these countries has frequently been mentioned in news coverage related to the immigration ban: The UK because of Brexit, Australia because of a leaked telephone call between President Trump and its Prime Minister, and Canada due to its shared border and its Prime Minister’s comments on welcoming refugees affected by the immigration moratorium.

Like our previous U.S. population survey, we used a nationally-representative sample of n=3000 for each of these countries.

Opposition Highest in Canada, Lowest in the UK

It probably does not come as a surprise to anyone who’s been following this issue in the media that citizens outside of America are less likely to approve of President Trump’s immigration moratorium.

I had honestly expected Australians to be the most strongly opposed to the order in light of the highly-publicized and problematic telephone call transcript leaked last week between President Trump and the Australian Prime Minister (which, coincidentally, involved a refugee agreement). But interestingly, people from our close ally and neighbor to the north, Canada, were most strongly opposed to the executive order (67%). The UK had proportionately fewer opposing the ban than Australia (56% vs. 60%), but the numbers of people opposed to the policy in both countries significantly lagged the Canadians. Emotions Run High Abroad Deriving emotions from text is an interesting and effective measure for understanding people’s opinions and preferences (and more useful than the “sentiment” metrics often discussed in text analytics and, particularly, in social media monitoring circles).

The chart below features OdinText’s emotional analysis of comments for each of the four countries across what most psychologists agree constitute the eight major emotion categories:

We can see that while the single highest emotion in American comments is joy/happiness, the highest emotion in the other three countries is anger. Canadians are angriest. People in the UK and Australians exhibit somewhat greater sadness and disgust in their comments. Notably, disgust is an emotion that we typically only see rarely in food categories. Here it takes the form of vehement rejection with terms such as “sickened,” “revolting,” “vile,” and, very often, “disgusted.” It is also worth noting that in cases, people directed their displeasure at President Trump, personally.

Examples:

"Trump is a xenophobic, delusional, and narcissistic danger to the world." – Canadian (anger) “Most unhappy - this will worsen relationships between Muslims and Christians.” – Australian (sadness) "It's disgusting. You can't blame a whole race for the acts of some extremists! How many white people have shot up schools and such? Isn't that an act of terror? Ban guns instead. He's a vile little man.” –Australian (disgust)

UK comments contain the highest levels of fear/anxiety:

"I am outraged. A despicable act of racism and a real worry for what political moves may happen next." – UK (fear/anxiety)

That said, it is also important to point out that there is a sizeable group in each country who express soaring agreement to the level of joy:

“Great move! He should stop all people that promote beating of women” – Australian (joy) “Sounds bloody good would be ok for Australia too!” – Australian (joy) “EXCELLENT. Good to see a politician stick by his word” – UK (joy) “About time, I feel like it's a great idea, the United States needs to help their own people before others. If there is an ongoing war members of that country should not be allowed to migrate as the disease will spread.” – Canadian (joy)

Majority of Canadians Willing to Take Refugee Overflow Given Canada’s proximity to the U.S., and since people from Canada were the most strongly opposed to President Trump’s executive order, this raised the question of whether Canadians would then support a measure to absorb refugees that would be denied entrance to the U.S., as Prime Minister Justin Trudeau appears to support.

(Note: In a Jan. 31 late-night emergency debate, the Canadian Parliament did not increase its refugee cap of 25,000.)

 

A solid majority of Canadians would support such an action, although it’s worth noting that there is a significant difference between the numbers of Canadians who oppose the U.S. immigration moratorium (67%) and the number who indicated they would be willing to admit the refugees affected by the policy.

When asked a follow-up question on whether “Canada should accept all the refugees which are turned away by USA's Trump EO 13769,” only 45% of Canadians agreed with such a measure, 33% disagreed and 22% said they were not sure.

Final Thoughts: How This Differs from Other Polls Both the U.S. and the international versions of this study differ significantly from any other polls on this subject currently circulating in the media because they required respondents to answer the question in a text comment box in their own words, instead of just selecting from options on an “agree/disagree” Likert scale.

As a result, we were able to not only quantify support and opposition around this controversial subject, but also to gauge respondents’ emotional stake in the matter and to better understand the “why” underlying their positions.

While text analysis allows us to treat qualitative/unstructured data quantitatively, it’s important to remember that including a few quotes in any analysis can help profile and tell a richer story about your data and analysis.

We also used a substantially larger population sample for each of the countries surveyed than any of the conventional polls I’ve seen cited in the media. Because of our triangulated approach and the size of the sample, these findings are in my opinion the most accurate numbers currently available on this subject.

I welcome your thoughts!

@TomHCAnderson - @OdinText

About Tom H. C. Anderson Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the "Four under 40" market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson.