Posts in Will it Unstructure?
NFL Players Taking a Knee is More Complex and Polarizing than We Think

If a topic is worth quantifying - it’s also worth understanding the 'Why’s' behind it I think I’ve seen at least 10 polls on the NFL Protest issue in the last week. As usual, these polls are simple structured questions, sadly usually not much more than a simple agree or disagree option. For example, the CNN poll released this weekend showing 49% Against 43% Supporting, (I’m not quite sure what happened to the remaining 8%). A PBS/Marist Poll had 48% of Americans saying protests were respectful and 46% disrespectful. Yet another, Seaton Hall Poll, had 84% supporting the players right to protest!

Which of these is correct??? It seems with structured questions, you really do Receive what you Ask for

It’s different when you allow people to answer whatever they want. As we have done here on the blog several times before, in order to challenge structured/forced-answer polling and explore the results we get when people are allowed to say anything they want about a topic. This weekend we asked 1,500 Americans 1 single question:

“Q. What are your thoughts about NFL Players 'Taking a Knee' during the national anthem? [Please elaborate why you are for or against what these players are doing]”

We utilized our text analytics software platform OdinText to classify responses into 3 groups. Those clearly against the players protesting (46%), those supporting the players taking a knee (33%), and those who did not take a clear position either way, i.e. they either did not care, understand, or had more mixed emotions on the topic (21%). We then weighted these groups based on location, age, and gender to as accurately as possible project onto the general US population. Post weighting comments fell out slightly more even, though still with a majority Against the protests (43%), over a third Supporting player protests (37%) and one in five not taking a side explicitly (21%).

NFL OdinText Chart 1

In past text analytics polls, we have seen differences from other mainstream structured polls which seem to indicate text polls can provide slightly different, more accurate proportions as they relate to actual behavior (our pre-election TextPoll indicated the Trump-Clinton upset ahead of time).

We believe this may partly be due to the fact that when you ask someone why they feel a certain way, there is additional cognition when answering. Of course the true beauty of text analytics lies in better understanding and quantifying the relative importance of the various Why’s.

The WHY's Behind The Opinions

Visualizing the comment data about WHY Americans actually feel the way they do about the protests quickly paints a picture of the sentiment and the reasons behind them.

OdinText nfl vizualization 2

Along the X axis from left (those against the protests in red) to right (those supporting the protests in green), with Y axis representing frequency of mentions, we see that “disrespectfulness” is the most frequently occurring theme, and it is indeed more frequently mentioned than the primary reason for supporting the protests on the right in green “freedom of speech”.

In the chart below we can see the most important reasons WHY fans agree or disagree with player protests even more clearly (sorted by difference gap).

NFL OdinText chart 3

Beyond looking at sentiment and what was answered, with text analytics we may even consider the mechanics of how questions were answered. Those who were not clearly for one side or the other, while answering fastest (49 seconds on average) and having shorter answers on average tended to use slightly longer/complex words. Those Against, while taking longest to answer (1.5 min on average) tended to have responses of more similar sentence and word length as those Supporting the player protests (though the latter group answered somewhat faster, in about 1 minute). This longer response time might give some indication on the relative importance of the issue to the group, i.e. those Against the protesters may take the issue more seriously/personally.

The most interesting thing here, though of course, are the specific topics mentioned by the two groups. Among those Against the players taking a knee, Disrespect for flag, Country and Armed service members was the most frequent issue mentioned. Several mentioned that they would be choosing not to watch Football/NFL. Among respondents who self identified as Veterans in their response, somewhat more were in the Against camp.

Freedom of Speech, and the Right to Peaceful Protest were the two most mentioned topics related to Support for player protests.

The two sides seem to differ most on the appropriateness of venue, with many of those disagreeing with the players, pointing out that they are well paid (over paid in fact), and that the work place/NFL/Entertainment is not the place for politics. The idea of appropriateness not just of venue, but respect for flag and country was also brought up frequently by those with more Mixed opinions.

Don't Forget to Drill Down Some

I would be remiss if I told you three simple charts were sufficient in order to understand the American Psyche on an emotionally charged issue such as this one. One of our users pointed this out quite elegantly in a guest post recently here.

While many times a good visualization or two are enough to communicate everything you need to know in a text analysis, other times, we do need to remind ourselves to give a few actual example quotes.

While obviously not in our data for this study, I believe Obama may have said things well on the NFL Protests before leaving office “I believe that us honoring our flag and our anthem is part of what binds us together as a nation. But I also always try to remind folks that part of what makes this country special is that we respect people's rights to have a different opinion…it is important for everybody to listen to each other. I want (the protesters) to listen to the pain that that may cause somebody who, for example, had a spouse or a child who was killed in combat and why it hurts them to see somebody not standing… But I also want people to think about the pain he may be expressing about somebody who's lost a loved one that they think was unfairly shot."

I’m leaving you below with about a dozen sample quotes (more than I usually would recommend) for each of the three groups we identified above [sic]. These days, perhaps more so than ever before, I think we really do need to take the time to listen to each other. I welcome your thoughts to this post below, and am curious if any of the specific example quotes below resonate with you and if so which one?

AGAINST

“stupid disrespectful to our soldiers these are paid men just do your job what if doctors and nurses stated their political views when you where going into surgery a person wouldn't care for that either”

“I'm completely against it as an immigrant i have always been in awe of america's patriotism in this political climate we need to stand strong together and love our country otherwise this country will not last”

“i think it disrespects the u.s the national anthem an the american flag and is an inappropriate expression of private political views in the workplace during the player's performance of his duties and disrespects the fact that the fans in the stands have paid their hard earned money to watch a football game not to be subjected to the political views of the players the fans aren't paying for their tickets so that players can express themselves at work”

“im all for free speech but not during our national anthem i find it completely disrespectful for the true heroes that chose not to sit down and fought for our country there is no debate it's classless and demeaning and a distraction to sports”

“I am not allowed to make political statements at my job they shouldn't at theirs either the NFL wouldn't let the cowboys wear a patch in support of police yet it will allow this madness I am done with the NFL for life”

“Just play football I don't tune in to have politics pushed in my face if i wanted to hear others political views I'd watch cnn not the NFL”

“wWhy not protest any cause that anyone thinks of, valid or not, have no order at public events everything wrong in the world should be protested not just selected causes not fair to just limit to a few causes”

“As a veteran I am completely against anyone who takes a knee or protests during our national anthem I understand people feeling a need to peacefully protest but during our national anthem is not the time”

“Counterproductive and self indulgent inevitably perceived as a poke in the eye to the USA protest doesn't illuminate the cause hence a failure”

“They have a right to protest I have a right to not watch and cancel my NFL package which I have done”

“iIm against it they need to focus on football and not do something that further puts our president in the spotlight they're only rewarding the very person they're trying to speak out against”

UNCLEAR/NO OPINION/MIXED

“Stupid controversy desperate president grasping for distractions”

“I don't believe I could care any less about NFL nor the national anthem and how to listen to it…”

“I am very mixed on this. Part of me thinks it is wrong and getting out of hand that our flag and national anthem represent the freedom we have from sacrifices of those who have fought and died to protect us and that kneeling represents lack of respect, but like it or not the freedom allowed to live in our democracy gives us the right to do things as long as people are not physically hurt or mentally crushed freedom to protest i cannot give better answer”

“There is a political process...this...and Mr President please stop using twitter and calling names that is not appropriate either it all causes strife... players want to make a statement then use the process”

“I am for their freedom of protest I am against using the NFL as a venue of protest leave politics out of the NFL”

“People have the right to express themselves people fought for the right to peacefully protest no one is against the us army or the flag sports and politics shouldn't mix either have players stop coming out of the locker room for the anthem or stop doing it all together at games”

“They certainly have the right to protest however sports does not need to be mixed with politics sports is an escape from that”

“I'm neither for or against the players what i am against is the president and his endless cruel remarks on any subject”

“As long as they decide what to do as a team who cares. Before the year 2000 players were never on the field for it anyway”

“I am a veteran part of me accepts that one of the things I left my family and home to do was to protect others freedom of speech on the other hand I think there are ways of protesting that don't show disrespect to the nation via its symbol the flag that gives you those rights and doesn't serve as a slap in the face to everyone who has saluted it fold it to hand to a devastated family member or had one of those folded flags handed to them”

“I fully support their 1st amendment right to do so and think it is absolutely ridiculous that they are expressing their outrage against a country whose unparalleled and unprecedented levels of personal freedom and protection have made it possible for them to become multi millionaires among 1 of the wealthiest people on earth in their 20s for playing a game the right to free speech means the right to say and support ridiculous things including ridiculously myopic grievances they should try living one month in the life of any ordinary citizen from basically anywhere in the southern hemisphere and see if they ever complain again about life in the united states but go ahead kneel sit lay down stand on your head whatever makes you feel”

SUPPORT

“They began protesting the plight in america of African Americans now the president has forced them to protest him it's not illegal to not stand hand on heart for the american anthem and it is their constitutional right to protest ills in our country any way they choose no one mentions but part of every oath for public office including president say you will defend the constitution the president needs to read it”

“We have freedom of speech period not being able to exercise that makes us no different than Russia or North Korea or others exercising our freedoms should be the highest honor to the military why else have they fought and died plus the flag represents America not the military not veterans the national anthem represents America not just the military and veterans the flag flies over government buildings it's carried by olympic teams it's hung on lawns it's a symbol of America and does not solely belong to patriotism or the military or veterans by requiring them to stand would be forced nationalism and the first step to losing our freedoms that veterans have fought for to require them to stand would be spitting in the face of those who died so that we can be truly free”

“I think they are courageous to use their position to express their concern about inequality in America as both the athletes and many veterans have expressed the kneeling is not about disrespect to the flag or the anthem sad that we must disagree about everything these days it seems that people would rather believe it is an act of disrespect than even try to understand the issue”

“I applaud it you don't have to agree with their stance to be glad to see people exercising their rights to free speech”

“It's an effective public protest against a president that solely focuses on public image at the cost of any substantial or meaningful actions”

“I'm for it but it should be for police brutality against all Americans it is not just blacks that are being harmed”

“It is their freedom of speech if we force people to stand we are no better than North Korea, China or Syria all places that force standing to their pledge”

“I think their stance is admirable and courageous taking a stand for those who have no voice”

“Ok they are like it or not role models and seem to be on justice's side and if the pres would shut up more we would all be better off”

“I am for the nflNFL players taking a knee our flag means nothing if your brother is not included or does not feel included is mistreated or feels mistreated we should address the needs of all citizens and not just a few”

“Only black men with a consistent platform to make a statement regarding racial inequality”

“It's their right this country takes everything too seriously the citizens of this country need to relax”

“Taking a knee is what sports players do to show respect when a fellow player is injured it is another way to show respect and courage”

“They are right to use their position in the spotlight to draw attention to social issues within our society the issues at hand are not about respecting our veterans or the flag itself and we should not be distracted by these arguments”

“You can't demand respect”

“It's their right why are public displays of patriotism necessary in the first place”

 

NOTE: This TextPollTM is a quick non client project intended to demonstrate unstructured data analysis using the OdinText analytics software. That said, results have a confidence interval of +/- 2.5% at the 95% confidence level (greater than the 3 outside polls mentioned at beginning of post). If this had been an actual study we would have recommended looking at data by political affiliation as well as NFL viewing.

About Tom H. C. Anderson Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose patented SaaS analytics platform is used by companies like Disney and Coca-Cola to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the “Four under 40” market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson. Click here to request a demo or additional info.

 

 

 

 

 

 

 

New Sentiment Analysis Reveals Reasons Behind Stance on Confederate Statues

Text Analytics Poll™ shows asking respondents to provide reasons for their opinions may increase cognition and decrease “No Opinion”

Asking People WHY They Support/Oppose Civil War Monuments May Affect Results. Judging from the TV news and social media, the entire country is up in arms over the status of Confederate Civil War monuments. What really is the mood of the country in regard to these statues?

A quick Google search turned up the chart below, which to YouGov’s credit broke out not just Democrats VS Republicans, but also blacks VS whites. On a high level this structured survey question, which allowed respondents to answer a standard five-point agreement scale from ‘strongly approve’ to ‘strongly disapprove,’ seems to indicate that “almost half of Americans (48%)” want the Charlottesville Robert E. Lee statue to stay.

While emotions as depicted on TV and social media are running  high, there doesn’t seem to be much reasoned discussion about WHY people feel so strongly on either side. Therefore, we were curious if rather than just asking a closed-ended agreement scale, what would happen instead if respondents were asked to elaborate on their choice with a reason?

Note: the goal here is not to uncover all the best reasons for or against keeping the statues. If that was the case we could approach a handful of social science professors with expertise in history, civil rights or ethics and psychology. Instead, we were curious to see if simply asking someone to consider a reason for their choice (even if they could not give a very good one) would affect the proportions of those agreeing or disagreeing. Of course, we were also curious about how many reasons each side might enumerate and what the quality of those reasons might be.

We asked a random sample of 1,500 Americans the following:

Q. Should Confederate Civil War Monuments be allowed in the US, why or why not?

Asking respondents to provide a reason, and using Text Analytics to measure sentiment, provided an almost identical number in favor of removing Confederate Civil War statues (29%) as the simple Likert scale poll; however, it halved the number of “Don’t Know/Don’t Care” responses (just 10%), apparently to the benefit of those who support keeping Confederate Civil War statues intact (61%).

EMOTIONS VS EXPLICIT REASONS

Let’s have a look at the reasons each side provided…

First, it’s noteworthy but not surprising that a number of the comments registered high emotional valence – especially anger – among both groups. Among those who favor keeping the statues, there is also significantly more fear/anxiety expressed in their comments.

As for the specific reasons, among those who want statues to remain, ‘history’ (implicitly the preservation of) is the most frequently mentioned reason by far (46%), and that history shouldn’t be deleted (3%), and history is both Good and Bad (2%).

The main argument among those who want to remove the statues is that Confederates were losers and traitors (9%) and that these statues should be limited to museums and battle grounds (8%), that glorifying what these men stood for is wrong (6%), as well as more general mentions of its symbolism of hate or slavery (6%).

A QUICK LOOK AT REGION

We took a quick look at answers by geography. Southerners were 5% more likely than total to mention the historic importance of the statues (35% VS 30% in total). They were also half as likely to have made the argument that statues for losers/traitors aren’t appropriate (1.7% VS 2.8% in total).

Americans in the Northeast region were significantly more likely than average to say they weren’t sure or didn’t care (7% VS 5% in total), and were also significantly more likely to mention the importance of “remembering” (3% VS 1% in total).

Americans in the West Region were significantly less likely to mention the importance of ‘History’ (25% VS 30% in Total).

The Verdict Changes When Asked Why

The court of public opinion in a standard Likert scale instrument appears fairly evenly split on whether or not to remover Confederate Civil War monuments, but when we ask people to explain why they hold a position on this matter in their own words, we see a significant shift in the data toward keeping these monuments intact.

Most respondents didn’t offer any surprises in terms of their explanations for why they support/oppose keeping the monuments. Indeed, a few arguments on both sides have already been fleshed out in the media, and this may have affected how people responded.

The ah-ha for us in this exercise was that the “don’t care/don’t knows” shrank by half when respondents were asked to provide a reason for their opinion. Whether this is a matter of causality, of course, is debatable. But it does suggest that allowing people to explain in their own words will produce a different, possibly more accurate picture, as well as which reasons have strongest appeal.

@TomHCAnderson

*Note: n=1,500 responses were collected via Google Surveys 8/19-8/21 2017. Google Surveys allow researchers to reach a validated U.S. General Population Representative sample by intercepting people attempting to access high-quality online content or who have downloaded the Google Opinion Rewards mobile app. Results are +/- 2.53% accurate at the 95% confidence interval. Data was analyzed using OdinText 8/21/17. Request more info on OdinText here.

About Tom H. C. Anderson Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the “Four under 40” market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson.

Text Analytics: It's Not Just for BIG Data

In a world focused on the value of Big Data, it's important to realize that Small Data is meaningful, too, and worth analyzing to gain understanding. Let me show you with a personal example. If you're a regular reader of the OdinText blog, you probably know that our company President, Tom Anderson, writes about performing text analytics on large data sets.  And yes, OdinText is ideal for understanding data after launching a rapid survey then collecting thousands of responses.

However for this blog post, I'm going to focus on the use of Text Analytics for smaller, nontraditional data set:  emails.

SMALL Data (from email) Text Analytics

I recently joined OdinText as Vice President, working closely with Tom on all our corporate initiatives. I live in a small town in Connecticut with an approximate population of 60,000.  Last year I was elected to serve our town government as an RTM member along with 40 other individuals.  Presently, our town's budget is $290M and the RTM is designing the budget for the next year.

Many citizens email elected members to let them know how they feel about the budget.  To date, I have received 280 emails. (Before you go down a different path with this, please know that I respond personally to each one -- people who take the time to write me deserve a personal response.  I did not and will not include in this blog post how I intend to vote on the upcoming budget, nor will I include anything about party affiliations. And I certainly will not share names.)

As the emails were coming in, I started to wonder … what if I ran this the data I was receiving through OdinText?  Would I be able to use the tool to identify, understand and quantify the themes in the people’s thoughts on how I should vote on the budget?

The Resulting Themes from Small Data Analytics

A note about the methodology:  Each email that I received contained the citizen's name, their email address and content in open text format.  Without a key driver metric like OSAT, CSAT or NPS to analyze the text against, I chose to use overall sentiment. Here is what I learned

Emails about the town budget show that our citizens feel Joy but RTM members need to recognize their Sadness, Fear and Anger

Joy:

“I have been a homeowner in Fairfield for 37 years, raised 4 kids here and love the community.”

Sadness:

“I am writing you to tell you that I am so unhappy with the way you have managed our town.”

Fear:

“My greatest concern seems to be the inability of our elected members to cut spending and run the town like a business”

Anger:

“We live in a very small house and still have to pay an absurd amount of money in taxes.”

Understanding the resulting themes in their own words

Reduce Taxes (90.16%)

“Fairfield taxes are much higher than surrounding communities.”

“Fairfield taxes are out of line with similar communities”

“The town has to stop raising taxes at such a feverish rate.”

“High taxes are slowly eroding the town of Fairfield.”

Moving if Taxes are Increased (25.13%)

“I am on a fixed income at 64, and cannot afford Fairfield’s taxes now. Please recognize that I cannot easily sell my house, due to the economy & the amount of homes on the market here”

“regret to say most of our colleagues and friends have an "exit strategy" to leave Fairfield”

“Our town is losing residents who are fed up and have moved or are moving to Westport and other towns with lower mil rates”

Reduce Spending (33.33%)

“... bring spending under control”

“Stop the spending please”

“... needs to trim fat at the local level, cut services, stop spending money”

“We need to keep taxes down as much as possible - even if it means spending cuts.”

Education ‘don’t cut’ (8.74%)

“… takes great pride in its education system”

“… promise of an excellent public education”

“… fiscal responsibility; however, not at the expense of the children and their right to an excellent education.”

Education ‘please cut’ (9.83%)

“Let's shave funding from all programs including education”

“... deeply questioning our education budget”

“... reduce the Education budget”

“I have a cherished budgetary item that I want protected--the library. Cut that last, after you cut education, police, official salaries”

Big Value from Small Data in Little Time

I performed this text analysis in 30 minutes. Ironically, it has taken me longer to write this blog post than it did to quantify the text from all those emails. Yet the information and understanding I have gleaned will empower me as I make decisions on this important topic. A small investment in small data has paid off in a BIG way.

Tim Lynch - @OdinText

What Americans Really Think about Trump’s Immigration Ban and Why

Text Analysis of What People Say in Their Own Words Reveals More Than Multiple-Choice Surveys It’s been just over a week since President Trump issued his controversial immigration order, and the ban continues to dominate the news and social media.

But while the fate of Executive Order 13769—“Protecting the Nation from Foreign Terrorist Entry into the United States”—is being hashed out in federal court, another fierce battle is being waged in the court of public opinion.

In a stampede to assess where the American people stand on this issue, the news networks have rolled out a parade of polls. And so, too, once again, the accuracy of polling data has been called into question by pundits on both sides of the issue.

Notably, on Monday morning the president, himself, tweeted the following:

Majority Flips Depending on the Poll

It’s easy to question the accuracy of polls when they don’t agree.

Although on the whole these polls all indicate that support is pretty evenly divided on the issue, the all-important sound bite of where the majority of Americans stand on the Trump immigration moratorium flips depending on the source:

  • NBC ran with an Ipsos/Reuters poll that found the majority of Americans (49% vs. 41%) support the ban.

 

 

  • CNN publicized results from an ORC Poll with the majority opposed to the ban (53% vs. 47%).

 

  • A widely reported Gallup poll found the majority of Americans oppose the order (55% to 42%).

 

There are a number of possible reasons for these differences, of course. It could be the way the question was framed (as suggested in this Washington Post column); it could be the timing (much has transpired and has been said between the dates these polls were taken); maybe the culprit is sample; perhaps modality played a part (some were done online, others by phone with an interviewer), etc.

My guess is that all of these factors to varying degrees account for the differences, but the one thing all of these polls share is that the instrument was quantitative.

So, I decided to see what if anything happens when we try to “unstructure” this question, which seemingly lends itself so perfectly to a multiple-choice format. How would an open-ended version of the same question compare with the results from the structured version? Would it add anything of value?

Part I: A Multiple-Choice Benchmark

The first thing we did was to run a quantitative poll as a comparator using a U.S. online nationally representative sample* of n=1,531 (a larger sample, by the way, than any of the aforementioned polls used).

In carefully considering how the question was framed in the other polls and how it’s being discussed in the media, we decided on the following wording:

“Q. How do you personally feel about Trump's latest Executive Order 13769 ‘Protecting the Nation from Foreign Terrorist Entry into the United States’ aka ‘A Muslim Ban’”?

We also went with the simplest and most straightforward closed-ended Likert scale—a standard five-point agreement scale. Below are the results:

Given a five-point scale, the most popular answer by respondents (36%) was “strongly disagree.” Interestingly, the least popular choice was “somewhat disagree” (6.6%).

Collapsing “strongly” and “somewhat” (see chart below) we found 4% more Americans (43%) disagree with Trump’s Executive Order than agree with it (39%). A sizeable number (18%) indicated they aren’t sure/don’t know.

Will It Unstructure? - A Text Analytics PollTM

Next, we asked another 1500 respondents from the same U.S. nationally online representative source* EXACTLY the same question, but instead of providing choices for them to select from, we asked them to reply in an open-ended comment box in their own words.

We ran the resulting comments through OdinText, with the following initial results:

As you can see, the results from the unstructured responses were remarkably close to those from structured question. In fact, the open-ended responses suggest Americans are slightly closer to equally divided on the issue, though slightly more disagree (a statistically significant percentage given the sample size).

This, however, is where the similarities between unstructured and structured data end.

While there is nothing more to be done with the Likert scale data, the unstructured question data analysis has just begun…

Low-Incidence Insights are Hardly Incidental

It’s worth noting here that OdinText was able to identify and quantify many important, but low-incidence insights—positive and negative— that would have been treated as outliers in a limited code-base and dismissed by human coders:

  • “Embarrassment/Shame” (0.2%)
  • “Just Temporary” (0.5%)
  • “Un-American” (0.9%)
  • “Just Certain/Specific Countries” (0.9%)
  • “Unconstitutional/Illegal” (2%)
  • “Not a Muslim Ban/Stop Calling it that” (2.9%)
  • ...

 

An Emotionally-Charged Policy

It shouldn’t come as a surprise to anyone that emotions around this particular policy run exceptionally high.

OdinText quickly quantified the emotions expressed in people’s comments, and you can see that while there certainly is a lot of anger—negative comments are spread across anger, fear/anxiety and sadness—there is also a significant amount of joy.

What the heck does “joy” entail, you ask? It means that enough people expressed unbridled enthusiasm for the policy along the lines of, “I love it!” or “It’s about time!” or “Finally, a president who makes good on his campaign promises!”

Understanding the Why Behind People’s Positions

Last, but certainly not least, asking the same question in an open-ended format where respondents can reply in their own words enables us to also understand why people feel the way they do.

We can then quantify those sentiments using text analytics and see the results in context in a way that would not have been possible using a multiple-choice format.

Here are a few examples from those who disagree with the order:

  • “Just plain wrong. It scored points with his base, but it made all Americans look heartless and xenophobic in the eyes of the world.”
  • “Absolutely and unequivocally unconstitutional. The foundation, literally the reason the first European settlers came to this land, was to escape religious persecution.”
  • “I don't like and it was poorly thought out. I understand the need for vetting, but this was an absolute mess.”
  • “I think it is an overly confident action that will do more harm than good.”
  • “I understand that Trump's intentions mean well, but his order is just discriminating. I fear that war is among us, and although I try my best to stay neutral, it's difficult to support his actions.”

Here are a few from those who agree:

  • “I feel it could have been handled better but I agree. Let’s make sure they are here documented correctly and backgrounds thoroughly checked.”
  • “I feel sometimes things need to be done to demonstrate seriousness. I do feel bad for the law abiding that it affects.”
  • “Initially I thought it was ridiculous, but after researching the facts associated with it, I'm fine with it. Trump campaigned on increasing security, so it shouldn't be a surprise. I think it is reasonable to take a period of time to standardize and enforce the vetting process.”
  • “I feel that it is not a bad idea. The only part that concerns me is taking away from living the American Dream for those that aren’t terrorists.”
  • “good but needed more explanation”
  • “OK with it - waiting to see how it pans out over the next few weeks”
  • “I think it is good, as long as it is temporary so that we can better vet those who would come to the U.S.”

And just as importantly, yet oft-overlooked those who aren’t completely sure:

  • “not my circus”
  • “While the thought is good and just for our safety, the implementation was flawed, much like communism.”

Final Thoughts: What Have we Learned?

First of all, we saw that the results in the open-ended format replicated those of the structured question. With a total sample of 3000, these results are statistically significant.

Second, we found that while emotions run high for people on both sides of this issue, comments from those who disagree with the ban tended to be more emotionally charged than from those who agreed with the ban. I would add here that some of the former group tended not to distinguish between their feelings about President Trump and the policy.

We also discovered that supporters of the ban appear to be better informed about the specifics of the order than those who oppose it. In fact, a significant number of the former group in their responses took the time to explain why referring to the order as “a Muslim ban” is inaccurate and how this misconception clouds the issue.

Lastly, we found that both supporters and detractors are concerned about the order’s implementation.

Let me know what you think. I’d be happy to dig into this data a bit more. In addition, if anyone is curious and would like to do a follow-up analysis, please contact me to discuss the raw data file.

@TomHCAnderson

Ps. Stay tuned for Part II of this study, where we’ll explore what the rest of the world thinks about the order!

*Note: Responses (n=3,000) were collected via Google Surveys. Google Surveys allow researchers to reach a validated (U.S. General Population Representative) sample by intercepting people attempting to access high-quality online content—such as news, entertainment and reference sites—or who have downloaded the Google Opinion Rewards mobile app. These users answer up to 10 questions in exchange for access to the content or Google Play credit. Google provides additional respondent information across a variety of variables including source/publisher category, gender, age, geography, urban density, income, parental status, response time as well as google calculated weighting. Results are +/- 1.79% accurate at the 95% confidence interval.

 

About Tom H. C. Anderson

Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the "Four under 40" market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson.

Top 2017 New Year’s Resolutions Text Analyzed (In Their Own Words)

Will it Unstructure? Part I of a New Series of Text Analytics Tests Happy New Year!

As I was preparing to celebrate the New Year with my family and pondering the year ahead, my mind wandered to all of those Top New Year’s Resolutions lists that you see the last week in December every year. It seems to me that the same resolutions with very similar incidence populate those lists each year, usually with something around diet and/or exercise as the most popular resolution.

After spending several minutes investigating, it occurred to me that these lists are almost always compiled using quantitative instruments with static choice answers pre-defined by researchers—therefore limited in options and often biased.

Here’s a good example of a study that has been repeated now for a few years by online financial institution GOBankingRates.com.

While their 2017 survey was focused solely on financial resolutions, their 2016 survey was broader and determined that “Live Life to The Fullest” was the most popular resolution (45.7%), followed by “Live a Healthier life” (41.1%) etc. [see chart below].

The question I had, of course, was what would this look like if you didn’t force people to pick from a handful of arbitrary, pre-defined choices?

Will It Unstructure?

You may be familiar with the outlandish but wildly popular “Will it Blend?” video series by Blendtec, where founder Tom Dickson attempts to blend everything from iPhones to marbles. It’s a wacky, yet compelling way to demonstrate how sturdy these blenders are!

Well, today I’m announcing a new series of experiments that we’re calling “Will it Unstructure?

The idea here is to take structured questions from surveys, polls and so forth we come across and ask: Will it Unstructure? In other words, will asking the same question in an open-ended fashion yield the same or different results?

(In the future, we’ll cover more of these. Please send us suggestions for structured questions you’d like us to test!)

 

Will New Year’s Resolutions Unstructure? A Text Analytics PollTM

So, back to those Top New Year’s Resolution lists. Let’s find out: Will it Unstructure?

Over New Year’s weekend we surveyed n=1,536 respondents*, asking them the same question that was asked in the GoBankingRates.com example I referenced earlier: “What are your 2017 resolutions?”

*Representative online general population sample sourced via Google Surveys.

Below is a table of the text comments quickly analyzed by OdinText.

As you can see, there’s a lot more to be learned when you allow people to respond unaided and in their own words. In fact, we see a very different picture of what really matters to people in the coming year.

Note: The GoBankingRates.com survey allowed people to select more than one answer.

Predictably, Health (Diet and/or Exercise) came in first, but with a staggeringly lower incidence of mentions compared to the percent of respondents who selected it on the GoBankingRates.com survey: 19.4% vs. 80.7%.

Moreover, we found that ALL of the top resolution categories in the GoBankingRates.com example actually appeared DRAMATICALLY less frequently when respondents were given the opportunity to answer the same question unaided and in their own words:

  • “Living life to the fullest” = 1.1% vs. 45.7%
  • Financial Improvement (make/save more and/or cut debt) = 2.9% vs. 57.6%
  • Spend more time with family/friends = 0.2% vs. 33.2%

Furthermore, the second most-mentioned resolution in our study didn’t even appear in the GoBankingRates.com example!

What we’ll call “Spirituality” here—a mix of sentiments around being kinder to others, staying positive, and finding inner peace—appeared in 8.3% of responses, eclipsing each of the top resolutions from the GoBankingRates.com example except diet/exercise.

After that we see a wide variety of equally often mentioned and sometimes contradictory resolutions. Now, bear in mind that some of these responses—“Drink more alcohol,” for example—were probably made tongue-in-cheek. Interestingly, even in those cases, more than one person said the same thing, which suggests it may mean something more. (I.e., could this have been filed under “Have Fun/Live Life to the Fullest”?)

These replies are all low incidence, sure, but they certainly provide a fuller picture. For instance, who would’ve predicted that “getting a driver’s license/permit” or “getting married” would be a New Year’s resolution? I would add that among these low incidence mentions, a text analysis a way to understand the relative differences in frequency between various answers.

Disturbingly, 0.3% (five people) said their 2017 resolution is to die. Whether or not these responses were in jest or serious is debatable. Our figure is coincidentally not so far off from estimates from reputable sources with expertise on the subject. For example, according to Emory University, in the past year approx. 1.0% of the U.S. population (2.3 million people) developed a suicide plan and 0.5% (1 million people) attempted suicide.

All of this said, obviously the GoBankingRate.com survey was not a scientific instrument. We selected it at random from a lot of similar “Top New Year’s Resolutions” surveys available.

These results are all, of course, relatively subject to interpretation and we can debate them on a number of fronts, but at the end of the day it’s unmistakably clear that a quantitative instrument with a finite set of choices tells an entirely different story than people do when they have the opportunity to respond unaided and in their own words.

Bonus: Top Three Most Important Events of 2016

Since the whole New Year’s resolutions topic is a little overdone, I ran an additional question just for fun: “Name the Three Most Important Things That Happened in 2016.”

Here are the results from OdinText ranked in order of occurrence in 2016..

If I had to answer this question myself I would probably say Donald Trump winning the U.S. Presidential Election, Russian aggression/hacking and Brexit.

But, again, not everyone places the same weight on events. So here’s yet another example of how much more we can learn when we ask people to reply unaided, in their own words.

Thanks for reading!

REMINDER: Let me know what questions you would like us to use for future posts on the “Will it Unstructure?” series!

Wishing you and yours a happy, healthy new year!

@TomHCAnderson

Tom H. C. Anderson

OdinText Inc. www.odintext.com

 

ABOUT ODINTEXT

OdinText is a patented SaaS (software-as-a-service) platform for advanced analytics. Fortune 500 companies such as Disney and Shell Oil use OdinText to mine insights from complex, unstructured text data. The technology is available through the venture-backed Stamford, CT firm of the same name founded by CEO Tom H. C. Anderson, a recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research. Anderson is the recipient of numerous awards for innovation from industry associations such as ESOMAR, CASRO, the ARF and the American Marketing Association. He tweets under the handle @tomhcanderson.