Posts tagged Net Promoter Score (NPS)
What story is your customer satisfaction data telling ABOUT YOUR BUSINESS

The Variance Mountain

TL;DR When looking at good companies the shape is a slope and in bad companies it looks like a mountain.

An often-overlooked dimension of this is the shape or distribution of the variance across topical areas, and the scores your customers have given you. When you look at the data, through this lens, things become apparent. For influential companies with happier customers, the shape of their variance is a slope while it looks like a mountain if your customers are unhappy.

4th Image Smaller.png
1st Image Smaller.png

Looking at this, it makes sense at a basic level. As Tolstoy proclaimed, "All happy families are alike; each unhappy family is unhappy in its own way." People that leave good reviews need to say something, and invariably it will be a lot of the same, things that go right go right for the same reasons. Whether through excellent service, a good product, or meeting a commitment, people will be happy in the same ways.

When it comes to things going wrong, though, they tend to go wrong differently.

Take a topic like Delays. Intuitively we know many people will be unhappy about a delay, but we also know that the negative of a delay will correspondingly provide a company with a chance to create a good impression by solving the problem. One high scoring comment read,

"Check-in was easy; the flight was delayed by about 20 minutes, so we assumed we would be arriving in Orlando about 20 minutes late... HOWEVER, our pilot put the engines in "afterburner," and we arrived 22 minutes early and about 5 minutes ahead of some severe thunderstorms!" 

This is a perfect example of a negative experience giving a company a chance to make a positive impression. It drives variance higher in negative topics and creates the upper left to lower right trend we see across industries and data sets. This trend is also present in our dataset assembled from over 5000 hotel reviews. This is true when charting the average CSAT score against the variance score.

This again depicts the sharp upper left to lower right trend, indicating that lower-scoring NPS topics have high variance and high NPS topics have low variance values. One high variance topics in the data was 'booking related comments,' and it's easy to see why booking problems can be an opportunity to overcome or fail, and why there is a wide discrepancy in scores.

"We booked a suite, and it was never given to us. They were so rude and refused to act as if they had a suite. After 6 hrs of travel, they did not care about their mistake. I would never stay here again."

"My wife and I arrived about 2 hours ahead of the check-in time, and they were completely booked, we had a reservation, but we were early. They found us a room, and it was amazing. They have valet that's owned by another company they are really expensive but worth it if you're driving like we were."

These examples highlight the natural variability of customers discussing booking related issues, and while all businesses strive to turn problems into opportunities, and it is natural not to bat a hundred percent on those opportunities. There is, however, a second possibility, when you have topics with low NPS and low CSAT scores that also have low variance. It would represent a more negative hypothesis that your company has issues where they are consistent and bad. This sort of chart would look different, more like a mountain than a ski slope. With this in mind, let's take a look at a quick plot of NPS for a telecom companies product (left chart).

This shows the telecom data with a standard OLS regression that basically tells us nothing due to the scattered nature of the data. However, when I break the data set in half at an NPS score of 5 and run two regressions, the results are starker.

This chart clearly shows their most negative topics in terms of NPS are also tied with the most positive topics for being the lowest in their variances. Not only are they doing things their customers dislike, but they are also consistent and repeatable with these actions.      

It represents the worst type of opportunity, a squandered one. 

 When plotting your variance data, it is essential to realize that high variance isn't always a bad thing; it can be a positive sign that the steps you have taken to address known problems are working. Remember, a low variance is not always good either; the only thing worse than being bad at something is being consistently bad at something!

Customer Satisfaction: What do satisfied vs. dissatisfied customers talk about?

Text Analytics Tips - Branding What do satisfied versus dissatisfied customers talk about? - Group Comparison Example Text Analytics Tips by Gosia

In this post we are going to discuss one of the first questions most researchers tend to explore using OdinText: what do satisfied versus dissatisfied customers talk about? Many market researchers not only seek to find out what the entire population of their survey respondents mentions but it is even more critical for them to understand the strengths mentioned by customers who are happy and the problems mentioned by those who are less happy with the product or service.

To perform this kind of analysis you need to first identify “satisfied” and “dissatisfied” customers in your data. The best way to do it is based on a satisfaction or satisfaction-related metric, e.g., Overall Satisfaction or NPS (Net Promoter Score) Rating (i.e., likelihood to recommend). In this example, satisfied customers are going to be those who answered 4 – “Somewhat satisfied” or 5 – “Very satisfied” to the Overall Satisfaction question (scale 1-5). And dissatisfied customers are those who answered 1 – “Very dissatisfied” or 2 – “Somewhat dissatisfied”.

Next, you can compare the content of comments provided by the two groups of customers (Group Comparison tab). I suggest you first select the frequency of occurrence statistic for your comparison. You can use a dictionary or create your own issues that are meaningful to you and see whether the two groups of customers discuss these issues with different frequency or you can look at any differences in the frequency of most commonly mentioned automatic terms (which OdinText has generated automatically for you).

Figure 1. Frequency of issues mentioned by satisfied (Overall Satisfaction 4-5) versus dissatisfied (Overall Satisfaction 1-2) customers. Descending order of frequency for satisfied customers.Figure 1. Frequency of issues mentioned by satisfied (Overall Satisfaction 4-5) versus dissatisfied (Overall Satisfaction 1-2) customers. Descending order of frequency for satisfied customers.

In the attached figure you can see a chart based on a simple group comparison using a dictionary of terms of a sample service company. There you go, lots of exciting insights to present to your colleagues based on a very quick analysis!


Text Analytics Tips with Gosi

[NOTE: Gosia is a Data Scientist at OdinText Inc. Experienced in text mining and predictive analytics, she is a Ph.D. with extensive research experience in mass media’s influence on cognition, emotions, and behavior.  Please feel free to request additional information or an OdinText demo here.]

How to Increase the Amount of Text Data for Analysis

Text Analytics Tips - Branding How to Increase the Amount of Text Data for AnalysisText Analytics Tips by Gosia

If you find yourself slightly disappointed by the quantity or quality of text comments provided by your respondents you are definitely not alone. This is a common problem especially when survey respondents are not compensated for their answers and when they are allowed to leave open-ended questions unanswered.

However, don’t give up and immediately start collecting more data or design a new survey. You current dataset may still contain valuable information in the form of text comments. A good practice is to pool together all text comments from a number of text variables in your dataset. You can select all of them or just a subset that makes the most sense to be analyzed together.

Pooling text data for a richer analysis.

Figure 1. Pooling text data for a richer analysis.

In the attached figure, the bubble on the left represents probably the most frequently analyzed question in customer satisfaction surveys – the open-ended question following a key rating (e.g., Overall Satisfaction Rating or Net Promoters Score Rating). Most of these surveys will have at least one or more very good questions that can compliment the answers given to the open-ended question on the left (see the remaining bubbles on the right of the figure). So why not analyze them altogether? To do that - simply merge these text variables in your data editor remembering to leave a blank space between the content of the columns you are merging.

Conclusion: Enriching your data can be simple and powerful.

This very simple pooling of text data from various open-ended questions will allow you to significantly enrich you analysis in OdinText.



Text Analytics Tips with Gosi

[NOTE: Gosia is a Data Scientist at OdinText Inc. Experienced in text mining and predictive analytics, she is a Ph.D. with extensive research experience in mass media’s influence on cognition, emotions, and behavior.  Please feel free to request additional information or an OdinText demo here.]

What Your Customer Satisfaction Research Isn’t Telling You and Why You Should Care

Why most customer experience management surveys aren’t very useful


Most of your customers, hopefully, are not unhappy with you. But if you’re relying on traditional customer satisfaction research—or Customer Experience Management (CXM) as it’s come to be known—to track your performance in the eyes of your customers, you’re almost guaranteed not to learn much that will enable you to make a meaningful change that will impact your business.Why Are Your Customers Mad At You-revise v2

That’s because the vast majority of companies are almost exclusively listening to happy customers. And this is a BIG problem.

Customer Satisfaction Distribution - Misconception: Most Customer Feedback is Negative

To understand what’s going on here, we first need to recognize that the notion that most customer feedback is negative is a widespread myth. Most of us assume incorrectly that unhappy customers are proportionately far more likely than satisfied customers to give feedback.

... the vast majority of companies are almost exclusively listening to happy customers. And this is a BIG problem.

In fact, the opposite is true. The distribution of satisfied to dissatisfied customers in the results of the average customer satisfaction survey typically follows a very different distribution. Indeed, most customers who respond in a customer feedback program are actually likely to be very happy with the company.

Generally speaking, for OdinText users that conduct research using conventional customer satisfaction scales and the accompanying comments, about 70-80% of the scores from their customers land in the Top 2 or 3 boxes. In other words, on a 10-point satisfaction scale or 11-point likeliness-to-recommend scale (i.e. Net Promoter Score), customers are giving either a perfect or very good rating.

That leaves only 20% or so of customers, of which about half are neutral and half are very dissatisfied.

So My Survey Says Most of My Customers Are Pretty Satisfied. What’s the Problem?

Our careful analyses of both structured (Likert scale) satisfaction data and unstructured (text comment) data have revealed a couple of important findings that most companies and customer experience management consultancies seem to have missed.

We first identified these issues when we analyzed almost one million Shell Oil customers using OdinText over a two-year period  (view the video or download the case study here), and since then we have seen the same trends again and again, which frankly left us wondering how we could have missed these patterns in earlier work.

1.  Structured/Likert scale data is duplicative and nearly meaningless

We’ve seen that there is very little real variance in structured customer experience data. Variance is what companies should really be looking for.

The goal, of course, is to better understand where to prioritize scarce resources to maximize ROI, and to use multivariate statistics to tease out more complex relationships. Yet we hardly ever tie this data to real behavior or revenue. If we did, we would probably discover that it usually does NOT predict real behavior. Why?

2.  Satisficing: Everything gets answered the same way

The problem is that customers look at surveys very differently than we do. We hope our careful choice of which attributes to measure is going to tell us something meaningful. But the respondent has either had the pleasant experience she expected with you  OR in some (hopefully) rare instances a not-so-pleasant experience.

The problem is that customers look at surveys very differently than we do. We hope our careful choice of which attributes to measure is going to tell us something meaningful.

In the former case her outlook will be generally positive. This outlook will carry over to just about every structured question you ask her. Consider the typical set of customer sat survey questions…

  • Q. How satisfied were you with your overall experience?
  • Q. How likely to recommend the company are you?
  • Q. How satisfied were you with the time it took?
  • Q. How knowledgeable were the employees?
  • Q. How friendly were the employees? Etc…

Jane's Experience: Jane, who had a positive experience, answers the first two or three questions with some modicum of thought, but they really ask the same thing in a slightly different way, and therefore they get very similar ratings. Very soon the questions—none of which is especially relevant to Jane—dissolve into one single, increasingly boring exercise.

But since Jane did have a positive experience and she is a diligent and conscientious person who usually finishes what she starts, she quickly completes the survey with minimal thought giving you the same Top 1, 2 or 3 box scores across all attributes.

John's Experience: Next is John, who belongs to the fewer than 10% of customers who had a dissatisfying experience. He basically straightlines the survey like Jane did; only he checks the lower boxes. But he really wishes he could just tell you in a few seconds what irritated him and how you could improve.

Instead, he is subjected to a battery of 20 or 30 largely irrelevant questions until he finally gets an opportunity to tell you his problem in the single text question at the end. If he gets that far and has any patience left, he’ll tell you what you need to know right there.

Sadly, many companies won’t do much if anything with this last bit of crucial information. Instead they’ll focus on the responses from the Likert scale questions, all of which Jane and John answered with a similar lack of thought and differentiation between the questions.

3.  Text Comments Tell You How to Improve

So, structured data—that is, again, the aggregated responses from Likert-scale-type survey questions—won’t tell you how to improve. For example, a restaurant customer sat survey may help you identify a general problem area—food quality, service, value for the money, cleanliness, etc.—but the only thing that data will tell you is that you need to conduct more research.

For those who really do want to improve their business results, no other variable in the data can be used to predict actual customer behavior (and ultimately revenue) better than the free-form text response to the right open-ended question, because text comments enable customers to tell you exactly what they feel you need to hear.

4.  Why Most Customer Satisfaction or NPS Open-End Comment Questions Fail

Let’s assume your company appreciates the importance of customer experience management and you’ve invested in the latest text analytics software and sentiment tools. You’ve even shortened your survey because you recognize that the be Overall Satisfaction (OSAT) and most predictive answers come from text questions and not from the structured data.

You’re all set, right? Wrong.

Unfortunately, we see a lot of clients make one final, common mistake that can be easily remedied. Specifically, they ask the recommended Net Promoter Score (NPS) or Overall Satisfaction (OSAT) open-end follow-up question: “Why did you give that rating?” And they ask only this question.

There’s nothing ostensibly wrong with this question, except that you get back what you ask. So when you ask the 80% of customers who just gave you a positive rating why they gave you that rating, you will at best get a short positive about your business. Those fewer than 10% who slammed you will give you  problem area certainly, but this gives you very little to work with other than a few pronounced problems that you probably knew were important anyway.

What you really need is information that you didn’t know and that will enable you to improve in a way that matters to customers and offers a competitive advantage.

An Easy Fix

The solution is actually quite simple: Ask a follow-up probe question like, “What, if anything, could we do better?”

This can then be text analyzed separately, or better yet, combined with the original text comment which as mentioned earlier usually reads Q. “Why did you give the satisfaction score you gave? And due to the Possion distribution in customer satisfaction yields almost only positive comments with few ideas for improvement.  This one-two question combination when text analyzed together fives a far more complete picture to the question about how customer view your company and how you can improve.

Final Tip: Make the comment question mandatory. Everyone should be able to answer this question, even if it means typing an “NA” in some rare cases.

Good luck!

Ps. To learn more about how OdinText can help you learn what really matters to your customers and predict real behavior,  please contact us or request a Free Demo here >


[NOTE: Tom H. C. Anderson is Founder of Next Generation Text Analytics software firm OdinText Inc. Click here for more Text Analytics Tips ]

Key Driver Analysis: Top-down & Bottom-up Approach

Text Analytics Tips - Branding Get a complete picture of your data: The ‘Top-Down and Bottom-Up Approach’

At OdinText we’ve found that the best way to identify all key drivers in any analysis really, especially in customer experience management (including but not limited to KPI’s such as OSAT, Net Promoter Score, Likelihood to Return or other real behavior) is through a dual process combining a theory-driven (aka “top-down”) and a data-exploratory or data-driven approach (aka “bottom-up”):


This approach requires you to identify important concepts or themes before even starting to explore and analyze your data. In customer satisfaction or brand equity research you can often start by identifying these key concepts by reviewing the strengths and weaknesses associated with your brand or product, or by listing the advantages and challenges that you believe may be prevalent (e.g., good customer service, poor management, professionalism etc.). This is an a priori approach where the user/analyst identifies a few things that they believe may be important.


This approach requires you to use a more advanced text analytics software, like OdinText, to mark and extract concepts or themes that are most frequently mentioned in customers’ text comments found in your dataset and that are relevant to your brand or product evaluation (e.g., high cost, unresponsiveness, love). Better analytics software should be able to automatically identify important things that the user/analyst didn’t know to look for.

Top-down vs. Bottom-up

The top-down approach does not reflect the content of your data, whereas the bottom-up approach while being purely based on the data can fail to include important concepts or themes that occur in your data less frequently or is abstracted in some way. For instance, in a recent customer satisfaction analysis, very few customer comments explicitly mentioned problems associated with management of the local branches (therefore, “management” was not mentioned frequently enough to be identified as a key driver by the software using the bottom-up approach).

However as the analyst had hypothesized that management might be an important issue, more subtle mentions associated with the concept of management were included in the analysis. Subsequently predictive analytics revealed that “poor management” was in fact a major driver of customer dissatisfaction. This key driver was only “discovered” due to the fact that the analyst had also used a top-down approach in their text analysis.

It may be that some of the concepts or themes identified using the two approaches overlap but this will only ensure that the most important concepts are included.

Remember, that only when combining these two very different approaches can you confidently identify a complete range of key drivers of satisfaction or other important metrics.

I hope you found today’s Text Analytics Tip useful.

Please check back in the next few days as we plan to post a new interesting analysis similar to, but even more exciting than last week’s Brand Analysis.


Text Analytics Tips with Gosi


[NOTE: Gosia is a Data Scientist at OdinText Inc. Experienced in text mining and predictive analytics, she is a Ph.D. with extensive research experience in mass media’s influence on cognition, emotions, and behavior.  Please feel free to request additional information or an OdinText demo here.]

Forbes: Text Analytics Gurus Debunk Four Big Data Myths

Four Big Data and Text Analytics Myths Debunked [Visit for more detailed article]

There are a lot of myths out there surrounding next generation research techniques around data from small to big. Text analytics, if done correctly, can offer insights into problems from a new more encompassing and accurate perspective.

OdinText CEO, Tom H. C. Anderson, and CTO Chris Lehew recently spoke to Forbes Magazine about some of the common analytics and text analytics myths they frequently encounter, and gave real examples of how OdinText users are benefiting from superior insights provided by the Next Generation Text AnalyticsTM platform.

The four data myths covered today in Forbes were:

Myth 1: Big Data Survey Scores Reign Supreme

 A myth propagated by Bain Consulting, without any evidence to justify the claims, is that a survey metric asking 'likeliness to recommend' is related to business growth. OdinText found absolutely no link between survey metrics like NPS and revenue when analyzing 800,000 customer surveys for Shell Oil/Jiffy Lube.

Actual customer text comments proved to be the single best predictor of real behavior from loyalty/returns to revenue.


Myth 2: Bigger Social Media Data Analysis Is Better


Many believe big data must have value simply because it's big - but it’s more about smart data than big data. Using OdinText Coca-Cola’s Social Media Hub found that only smarter text analytics and data translate to better insights.


Myth 3: New Data Sources Are The Most Valuable

 Many companies go looking for new data on social sites and elsewhere while failing to first look too what valuable data they may already be collecting . Campbell’s Soup for instance realized that they could get better insights by using OdinText to listen to the quarter of a million customers that contact their customer service department each year than relying on anything found on Twitter.


Myth 4: Keep Your Eye On The Ball By Focusing On How Customers View You

 Sometimes online discussion boards and other consumer generated media is the best place to find insights. But companies often focus just on what is said about them. Starwood hotels among others have realized the benefit of doing comparative analysis VS. their competition, arguably the single best use of this type of data.


You can read more detail about the four data myths discussed above in Forbes Magazine article here.

Please visit our blog in the coming months as we roll out a new series of “Text Analytics Tips” with experts from OdinText discussing more myths and sharing tips and techniques on how to drive better more actionable insights through smarter Next Generation Text AnalyticsTM

When Text Analytics is Your Brand

When Text Analytics is Your BrandWhat I learned about personal branding at IIEX

Coming back from Insight Innovation Exchange (IIEX) this week in Atlanta and thought I’d blog briefly about the two panel sessions on Personal/Digital Branding in which I participated.


Text Analytics

My main reason for attending IIEX was actually to give a brief presentation on the dramatic improvements we've made to our OdinText text analytics software, and how it brings value to untapped consumer text data (open-ends, NPS reasons, customer feedback, website comments, etc.), and how it can really turn any market research analyst into a powerful Data Scientist. Because of IIeX’s stellar reputation, this was the first time we’ve ever given any kind of demo of OdinText in public. Usually our presentations are approved case studies about how our clients like Coca-Cola, Disney, Shell Oil, etc. are using the tool. Also, as text analytics remains a very competitive field, we prefer to share details around the software with those we know have the kind of data where OdinText can be useful.


However, since we are launching a new version of OdinText and I was assured by Lenny Murphy that, contrary to what I believed, most attendees actually want to see software demos rather than just hear use cases. In case you missed it, I've posted a brief teaser video below, along with a shameless plug before I go on. If you regularly collect comment type text data, we’d love to hear from you and get you more info about OdinText (Request Info Here). Shameless ad plug over.




Personal Branding

Other than showing off OdinText though, I was also honored to be asked to sit on a personal branding panel with prolific market research tweeters Tom Ewing and Annie Pettit, as well as Dave McCaughan who is a well-known name in East Asian and Australian market research circles.

On the Summer Friday (at 5:30pm no less) before our Monday morning session, Annie Pettit came up with the idea to field an impromptu convenience sample survey, and to my surprise by Sunday afternoon we already had about 150 comments relating to the panelists. Lenny Murphy who has also accumulated a loyal #MRX following on Twitter and on the Greenbook blog was also included in the survey which asked something like “Q. What three things first come to mind when you hear each of these names/personal brands?”.

Though this sample is a bit on the small side for OdinText I quickly visualized the comments to give us some idea of how similar/different each of these 5 ‘brands’ are and what specific topics most frequently co-occur with each of them.


I’m sure all of us were equally interested in the findings, because let’s face it, while EVERYONE has a personal brand (even if unfortunately not everyone recognizes it), few of us ever get an insight into what it really means to people in this unaided top-of-mind market research sort of way.

We agreed not to share any of each other’s raw data, but I’m fine sharing the first 40 responses I received (both good, bad and ugly) below, sorted alphabetically:

American linked in conversationalist

Analytical, ever-present, helpful analytics omnipresent

analytics geek


beard, omnipresence and self publicity



Cool Guy


Fun honest text analytics

Hans Christian Anderson

He's all about new, cool & hip in the quant world

His banner ads pursue me remorselessly around the web marketing



know his name but can't recall...

Lover of anything that reminds him of the Swedish socialist utopia



next gen guy

odin text - text pro

OdinText Text Analytics, smart, trustworthy




respected, helpful, innovative smart

Self promoter

Social media junkie

straight shooter. willing to challenge hyped claims. maybe falling too in love with his own methodology

text analysis

text analytics

text analytics odintext

Text analytics pro

Text Analytics, expert, outspoken, industry leader,

text analytics, NGMR, vikings

Text master, text Analytics

The first to advocate Next Gen Market Research, especially Text Analytics and Data Mining,

The first market researcher to truly understand social, AND bold enough to stand up against trade orgs on behalf

of mid-small research firms. A true research hero

Tom is a great example of focusing on one thing you really care about and want to make better,

and then actually doing that..

Tweeted this survey

up against trade orgs on behalf of mid-small research firms. A true research hero.

A first thing that struck me looking at both the responses for my ‘brand’ as well as those of the others on the panel was that the negative comments, while few overall, were also rather consistent proportionately across all of us.

I think this may have come as a surprise to some of the others, but I expected a few negative remarks related to some of the positions I’ve taken about market research. While I believe the majority of US researchers agree with me, my positions weren’t as welcome by an outspoken few researchers more closely associated or working for these trade organizations. So the question is, as it relates to our personal brands, should we shy away from controversy (as long as it’s not personal or destructive in nature)? And the answer is, I don’t think it’s hurt my brand at all; controversy often leads to change, and usually change for the better. I'm happy to be associated with these issues, and do not fear ruffling feathers.

Of greater importance, and more surprising to me, was that our company brands were almost never mentioned for any of us. I’ve been concerned whether my comments related to other areas of consumer insights research have taken away from what I really want to be known for, OdinText and Text Analytics. The good news was that when market researchers who know me think of me, they think “Text Analytics”. The bad news was that few mention the brand OdinText. But how bad is this really?

A few months ago I wrote about personal branding and Kristin Luck (someone else whom I definitely think should also have been on the panel). You can read that piece here, however, I think the main point is that personal brands undoubtedly create a different and more complex association network in the minds of people than corporate brands or logos do.


This can’t be a bad thing, I believe they are complimentary. If people think Tom H. C. Anderson = Text Analytics, they also are likely to think Text Analytics = Tom H. C. Anderson, and so when they have a need for text analytics, some will think of me, and then OdinText (even if the brand OdinText doesn’t first come to mind).

I’m not sure what the association network is for uber personal brands like Bill Gates or the late Steve Jobs, but I would venture to guess it’s similar. Surprisingly perhaps, Microsoft and Apple may well not be the first thing that comes to mind when someone first thinks about these two individual brands. Both really are far more complex than either of the company brands Microsoft and Apple. The individuals stand for so much more (philanthropy, design, success, strength, perseverance, intelligence, innovation…).

Definitely an interesting area, and one that could use more research, aided by text analytics of course, and OdinText ideally .

My takeaway and advice to other market researchers is that personal branding is a good thing. It’s a complex thing, and that’s a good thing. Unlike a simple company product or logo, we as people are deeper and have ability to encompass far more, and deeper dimensions. I believe these personal brands, as I know from experience is the case for both myself and Kristin Luck, have been very beneficial to the companies associated with us. It’s a truism, that this is a people business, and people buy from people.

I encourage everyone to give some thought to their personal brands. Unlike corporate brands they don’t have to be perfect. If they were, they would be very boring and one dimensional. Just be you – and let others know it!



[Tom H. C. Anderson is Founder & CEO of Text Analytics SaaS firm OdinText ( He tweets under @TomHCAnderson, blogs at and manages one of the largest and most engaged market research groups on LinkedIn, Next Gen Market Research.]

Text Analytics for 2015 – Are You Ready?

OdinText SaaS Founder Tom H. C. Anderson is on a mission to educate market researchers about text analytics  [Interview Reposted from Greenbook]

TextAnalyticsGreenbookJudging from the growth of interest in text analytics tracked in GRIT each year, those not using text analytics in market research will soon be a minority. But still, is text analytics for everyone?

Today on the blog I’m very pleased to be talking to text analytics pioneer Tom Anderson, the Founder and CEO of Anderson Analytics, which develops one of the leading Text Analytics software platforms designed specifically for the market research field, OdinText.

Tom’s firm was one of the first to leverage text analytics in the consumer insights industry, and they have remained a leader in the space, presenting case studies at a variety events every year on how companies like Disney and Shell Oil are leveraging text analytics to produce remarkably impactful insights.

Lenny: Tom, thanks for taking the time to chat. Let’s dive right in! I think that you, probably more so than anyone else in the MR space, has witnessed the tremendous growth of text analytics within the past few years. It’s an area we post about often here on GreenBook Blog, and of course track via GRIT, but I wonder, is it really the panacea some would have us believe?

Tom: Depends on what you mean by panacea. If you think about it as a solution to dealing with one of the most important types of data we collect, then yes, it can and should be viewed exactly that way. On the other hand, it can only be as meaningful and powerful as the data you have available to use it on.

Lenny: Interesting, so I think what you’re saying is that it depends on what kind of data you have. What kind of data then is most useful, and which is not at all useful?

Tom: It’s hard to give a one size fits all rule. I’m most often asked about size of data. We have clients who use OdinText to analyze millions of records across multiple languages, on the other hand we have other clients who use it on small concept tests. I think it is helpful though to keep in mind that Text Analytics = Text Mining = Data Mining, and that data mining is all about pattern recognition. So if you are talking about interviews with five people, well since you don’t have a lot of data there’s not really going to be many patterns to discover.

Lenny: Good Point! I’ve been really impressed with the case studies you’ve releases in the past year or two on how clients have been using your software. One in particular was the NPS study with Shell Oil. A lot of researchers (and more importantly CMOs) really believed in the Net Promoter Score before that case study. Are those kinds of insights possible with social media data as well?

Tom: Thanks Lenny. I like to say that “not all data are created equal”. Social media is just one type of data that our clients analyze, often there is far more interesting data to analyze. It seems that everyone thinks they should be using text analytics, and often they seem to think all it can be used for is social media data. I’ve made it an early 2015 new year’s resolution to try to help educate as many market researchers as I can about the value of other text data.

Lenny: Is the situation any different than it was last year?

Tom: Awareness of text analytics has grown tremendously, but knowledge about it has not kept up. We’re trying to offer free mini consultations with companies to help them understand exactly what (if any) data they have are good candidates for text analytics.

Lenny: What sources of data, if any, don’t you feel text analytics should be used on?

It seems the hype cycle has been focused on social media data, but our experience is that often these tools can be applied much more effectively to a variety of other sources of data.

However, we often get questions about IDI (In-Depth-Interviews) and focus group data. This smaller scale qualitative data, while theoretically text analytics could help you discover things like emotions etc. there aren’t really too many patterns in the data because it’s so small. So we usually counsel against using text analytics for qual, in part due to lower ROI.

Often it’s about helping our clients take an inventory around what data they have, and help them understand where if at all text analytics makes sense.

Many times we find that a client really doesn’t have enough text data to warrant text analytics. However this is sad in cases where we also find out they do a considerable amount of ad-hoc surveys and/or even a longitudinal trackers that go out to tens of thousands of customers, and they’ve purposefully decided to exclude open ends because they don’t want to deal with looking at them later. Human coding is a real pain, takes a long time, is inaccurate and expensive; so I understand their sentiment.

But this is awful in my opinion. Even if you aren’t going to do anything with the data right now, an open ended question is really the only question every single customer who takes a survey is willing and able to answer. We usually convince them to start collecting them.

Lenny: Do you have any other advice about how to best work with open ends?

ODIN AD 1 300X250

Tom: Well we find that our clients who start using OdinText end up completely changing how they leverage open ends. Usually they get far wiser about their real estate and end up asking both less closed ended questions AND open ended questions. It’s like a light bulb goes off, and everything they learned about survey research is questioned.

Lenny: Thanks Tom. Well I love what your firm is doing to help companies do some really interesting things that I don’t think could have been done with any other traditional research techniques.

Tom: Thanks for having me Lenny. I know a lot of our clients find your blog useful and interesting.

If any of your readers want a free expert opinion on whether or not text analytics makes sense for them, we’re happy to talk to them about it. Best way to do so is probably to hit the info request button on our site, but I always try my best to respond directly to anyone who reaches out to me personally on LinkedIn as well.

Lenny: Thanks Tom, always a pleasure to chat with you!

For readers interested in hearing more of Tom’s thoughts on Text Analytics in market research, here are two videos from IIeX Atlanta earlier this year that are chock full of good information:

Panel: The Great Methodology Debate: Which Approaches Really Deliver on Client Needs?

Discussing the Future of Text Analytics with Tom Anderson of Odin Text

The N in Text Analytics: Text Mining with Different Sample Sizes

 [Interview Reposted with Permission From Jeffrey Henning's ResearhAccess] runes of old

I recently had the opportunity to interview Tom H. C. Anderson, the founder of Anderson Analytics, about his ongoing application of text analytics to market research.

Q: What’s the process for optimally using text analytics with survey verbatim responses?

A: Well, that patented process is something that we’ve obviously put a lot of time and thought into with OdinText, and something that continues to evolve.

Generally speaking though I can say it’s important to look beyond the individual sentences, and not to get wrapped up in linguistically derived sentiment. The mistake I see being made most often is that text analytics is approached as a replacement for human coding. In our view they are apples and oranges. Yes, text analytics can replace human coding. But coding is just a small part of what we do: our real focus is on analytics, and often that means that the optimal use of verbatim responses is predictive analytics. That is the optimal use of survey verbatims.

Q: Is there a minimal sample size this makes sense for?

A:  I wouldn’t say that there’s a minimum size per se, though I would say that the ROI of text analytics increases exponentially with the size of the data. In our point of view “Natural Language Processing”, “Text Analytics”, “Text Mining” and even “Data Mining” are all synonyms, the last two of which are a better description of the process. What that means is that without a certain minimum size of data there will be no meaningful patterns to find (to mine).

Focus group data generally is not suitable for text analytics. It’s partly because the n is so small. But also because — although they can produce a large amount of text in total — this text is heavily influenced by the moderator. It very much depends on the data though. The smallest data size ever looked at in OdinText had sample size of n=2. This was the Obama/Romney debates, and each candidate spoke for about 45 minutes. More typically, though, text analytics is used to analyze tens of thousands, or hundreds of thousands, of records. These data are either from customer satisfaction/loyalty survey trackers, customer service center telephone transcripts or emails, or yes, social media.

Many of our customers do find text analytics useful for smaller ad-hoc survey data with sample sizes around n = ~1,000 as well. Once you are up and running with text analytics, it’s very easy and fast to use text analytics to get insights from data such as this. But you are somewhat more limited with the kinds of analysis that you can do with these smaller data sets. But if you do enough of these ad-hoc projects, text analytics can certainly provide relatively good ROI here too.

Q: Is it better suited for tracking studies rather than one-off surveys?

A: Better ROI with bigger better data. If you only do 5 to 10 ad-hoc surveys per year with an average of n=300, then text analytics may not be worth it.  As you move beyond this, it becomes more and more valuable.

Q: My initial impression after first hearing about your NPS work was simply that you improved the value of the survey by adding text analytics. But it seems like you are really about a holistic process, using CRM data and other information to build a predictive model. What are the data sources that you find produce the best value? While I think of Odin Text as text analytics, is it actually a predictive analytics solution whose differentiation is its text analytics capabilities?

A: Well, yes, you are right that OdinText is a text analytics system. We are not trying to become the next SAS or SPSS, per se; both of them have some good packages for basic statistics. Where OdinText is best is when there is also text data, and when the data gets bigger. Our clients are often working with data sets so large that they would take too long to run or more typically crash SPSS and the like. Working with text data requires more computing power. That’s something we are able to offer through our SaaS model.

In the case you mentioned, Shell was using OdinText to analyze their n = ~400,000 Jiffy Lube Net Promoter survey data. We suggested that they add some data from their CRM database, so they added actual behavioral data: visits as well as sales.

This is a unique strength to OdinText. We don’t believe it makes much sense to analyze text in isolation. We are building more analytics capabilities into OdinText currently.

Q: The text analytics space is very crowded — I’ve personally look at over 20 platforms. What sets Odin Text apart from other systems?

Three things, really all tied to our patented approach to text analytics:

  1. The way we allow you to use mixed data not just text.
  2. The way we filter our ‘noise’ and alert the analyst to things they might not have considered.
  3. And finally, our approach, while powerful, is also intuitive. We recognized early on that most clients don’t have any relevant training data, and when they do, using it to build models would just be mimicking inferior human coding. So unlike other enterprise solutions that require a lot of custom set up, our approach was developed to work very well off the shelf: it’s far more nimble in being able to deal with different data sources.

Jeffrey Henning, PRC, is president of Researchscape International, which provides “Do It For You” custom surveys at Do It Yourself prices.  He is a Director at Large on the Marketing Research Association’s Board of Directors. You can follow him on Twitter @jhenning.