Posts tagged social media
Text Analytics Poll: Why We Unfriend on Facebook

You Can't Handle the Truth (and Other Top Reasons for Being Unfriended on Facebook)  When’s the last time you unfriended someone on Facebook?

In spite of the fact that both the platform and its users have reached what in product lifecycle-speak would be called maturity, unfriending remains a somewhat surprisingly common phenomenon.

In fact, about 76% of Facebook users we contacted in a recent Text Analytics Poll™ told us they had unfriended someone .

The drivers behind “unfriending” have been researched and written about extensively, but given its enduring prevalence, we wanted to explore why it persists. So we asked a random gen pop sample of people (n=1500) the following:

“Have you unfriended someone on Facebook, and if so why?”

A brief note about this Text Analytics Poll™…

Before we share the results, I must emphasize that this is a quick, surface-level read. The goal of these Text Analytics Polls™ is not to conduct a perfect study, but to very quickly design and field a survey with only one open-ended question, analyze the results with OdinText, and report the findings here on this blog.

Yes, I know that not everyone is a Facebook user, and the response data reflected this. More importantly, there is without a doubt overlap among the categories that came out of OdinText’s analysis in the chart below.  For example, a number of people reported they had unfriended someone because they had stopped speaking to him/her. The reason they stopped speaking to this person could be due to a break-up, political differences or one of the other reasons cited by other respondents.

The point here is that people are responding off the top of their heads and in their own words, providing a much more accurate understanding of the “what” being explored, which we can then analyze and quantify using OdinText software.

Now for the results…

Top Reasons for Unfriending Someone on Facebook

Note: All Verbatims are [sic]

  1. Inappropriate/Offensive Content

You’d think most people would’ve learned appropriate Facebook etiquette by now, but clearly there are a significant number of bad actors out there, with the primary reason for unfriending being related to inappropriate or offensive content. This category spanned everything from profanity to racism to graphic sexual content, but also lifestyle choices or social views that people found personally objectionable.


they posted things that are illegal and go against my morals


R-rated content

Posts about lifestyle choice that I am against and profanity

constant postings of very controversial subjects such as abortion

racism or derogatory remarks about ladies

For posting an extremely offensive video featuring meth being smoked by young girls

  1. No Longer Friends/Grew Apart/Tired of Them

Too much drama or TMI, the airing of dirty laundry, posts that made people uncomfortable because they were too personal or too often about deeply sensitive subjects…these were common reasons cited for severing ties on Facebook. But people also said the relationship had grown stale or they’d lost touch. Sure people change. Relationships grow stale. Sometimes we outgrow one another. These are common occurrences in real life, but I was surprised to see such a high incidence of them on Facebook, where one’s “friends” are frequently not people with whom we share close personal relationships.


people who I fell out of contact with and felt it wouldn't be worth reconnecting with; people who I wasn't very fond of to begin with; people who were friended as a means to an end i.e. for an old job; people who I started out having a good friendship with and ended up being jerks.

because the friendship ended.

they were two-faced

No longer keep in touch

They changed as a person

I haven't seen them in years, and they tried to add me to a group about them selling Tupperware.

I had not seen or talked to them in months or years

They were posting too much personal drama

No longer friends with that person and couldn’t stand their drama anymore

They enjoyed high drama, public arguments with their families.

  1. Politics

Everyone knew this one was coming. We were surprised it wasn’t number one, especially given the particularly divisive presidential election that just took place. I expect the incidence of politics being a driver for unfriending is actually higher than people’s comments reflected. This is also an area where the comments were highly emotionally charged.

Interestingly, most people who cited politics as a reason for unfriending did not elaborate. The comment data suggest that a lot of people simply don’t care for political ranting, in general. It’s probably safe to assume that the rules that govern polite conversation in person also apply on Facebook.

That said, significant numbers of people did specify whether the politics of the unfriended were liberal or conservative. But interestingly, mentions of Donald Trump specifically were more than sufficient to merit their own category.

  1. Annoying Statuses

This one’s pretty self-explanatory: status updates—especially the inane and/or frequent variety—tend to irk people!


posted too many annoying statuses

Annoying posts that clutter the newsfeed

Because their posts were too frequent and annoying.

To stop the annoying posts

Annoying statuses

they either post way too much or their posts are stupid and I do not want to look at them

Tired of the persons constant posts about their hair or personal injuries from others

  1. Don’t Know Well or In Real Life

Early on in our Facebook lives, most of us were probably a bit promiscuous when it came to “friending” and, consequently, we had to cull the herd. What’s surprising is how many people still accept friends they don’t know and then “unfriend” them because…well…they don’t know them.

Did not feel comfortable with them knowing personal details about me

Didn't remember who they were.

I never knew them very well and they were cluttering my feed.

To purge people that I have no connection to

Because i didnt actually know them in real life.

Inactive, wasn't a close friend to begin with.

Because I did not speak to them ever

Because they were a mere acquaintance

cuz dont know the person

Bonus: Why We Think They Unfriended Us

It seems that while many of the norms and conditions for friendship (or at least friendly acquaintanceship) in real life also apply on Facebook, a lot of people either do not understand them or insist on breaking them when they’re on the other side of a screen.

But how do we account for those rare instances when someone unfriends us? Well, we asked another gen pop sample if they had been unfriended and, if so, for what reason.

Not surprisingly, it turns out we are almost 10 times more likely to say we unfriended someone because they were annoying or inappropriate as we are to admit that was the reason we ourselves were unfriended.

Most of the people who were unfriended by someone else told us they’re not quite sure why, but they often hazarded what appear to be pretty good “guesses”:

Yes, but I was not told why. Person has had a lot of stress in the community, can only guess it was just too much to continue.

I would guess that it would be because we don't keep in contact.

They did not like my ‘opinion’ which is fine because it was my ‘opinion’ not theirs. Guess they should not have asked.

Because of a joke that I made too soon after the Orlando shootings. I made a picture of Lando Calrissian holding a pride flag and the caption said ‘we are Lando?’ too soon I guess

People also tend to be rather defensive about being unfriended and are significantly more likely to just chalk it up to “a difference of opinion,” noting that they “don’t really care anyway” and that it’s probably because “we aren’t really close enough anymore.”

Finally, many of us say we were unfriended for “speaking the truth.”

Thanks for reading. Now what do you think?


PS. Do you have an idea for our next Text Analytics Poll™? We’d love to hear from you! Please contact us here for more info or to request an OdinText demo with your data.

About Tom H. C. Anderson

Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the "Four under 40" market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson.

Big Boulder Initiative Tackles Social Media Analytics!

Forging the Future of Social Analytics & Big Data in Colorado Next Week

I’m really looking forward to The Big Boulder Conference in Boulder, CO next week. This is a must-attend event for anyone with a stake or even an interest in social data.

Big Boulder Intitiative

Big Boulder Intitiative

For those of you not familiar with it, the conference is produced by The Big Boulder Initiative (BBI) — the first trade organization for the social media and data monitoring industry.

OdinText joined BBI two years ago at the urging of clients and partners who were excited to help establish learning and norms for better social analytics.

Next week’s event brings together some leading experts and top companies within the social data ecosystem to collectively address key challenges that face the industry.

One of the biggest problems with social analytics as I see it is quality of data. This is what I’m most interested in discussing and understanding in terms of what the future might hold and what we can do to clean and enrich what we already have.

If you’re planning on being there please say hello!

Tom H.C. Anderson | @TomHCanderson@OdinText

Tom H.C. Anderson

Tom H.C. Anderson

To learn more about how OdinText can help you understand what really matters to your customers and predict actual behavior,  please contact us or request a Free Demo here >

[NOTE: Tom H. C. Anderson is Founder of Next Generation Text Analytics software firm OdinText Inc. Click here for more Text Analytics Tips]

Attensity, Clarabridge vs. OdinText: What’s the Difference?

Attensity Clarabridge Text Analytics Software Comparison - The Printing Press Still Prints, But Who Would Want To? I’m always a bit reluctant to talk about competitors because I don’t want to disparage anyone, but people often ask me: what differentiates OdinText from your two big, well-known text analytics software competitors, Attensity and Clarabridge?


Attensity and Clarabridge are traditional text analytics tools, but they adhere to an outmoded, rules-based approach. This means they require costly and time-consuming expert customization before they can be useful to a client.

Furthermore, once these rules-based dictionaries are created, they only apply to the data used to create the rules. So, if you attempt to use the tool in another industry, category or company or for a different data set, critical exceptions to these rules creep up that render them useless.

Attensiity Clarabridge Text Analytics Software Comparison


In contrast, we built OdinText from an analyst’s perspective—not a developer’s—so that it’s intuitive, adaptive, data agnostic and fast. It doesn’t need all of this extensive priming and it works great right out of the box, which cuts the speed to insight dramatically.

The platform is easy to use, trainable to everyone and flexible in order to provide long-term value across an organization. This is part of the reason why we refer to our solution as Next Generation Text AnalyticsTM.


OdinText is the culmination of more than a decade of applied text analytics experience as a user of multiple text mining software platforms for large clients, including social media giants like Facebook and LinkedIn.

We realized that all of these platforms were built on an approach that required custom dictionaries and linguistic rules—they are more similar than different—and on the analytics side they all lacked fundamental capabilities to perform the tasks for which researchers like us needed them.


The exclusive advantages to OdinText’s empirically-based, patented approach include what we refer to as Contextual Sentiment and ESC (Noise Reduction). Put simply, OdinText automatically filters out noise and brings important verbatim issues and relationships in the data to the user’s attention, allowing them to easily discover what they may not otherwise even have known to look for.

[Contact us for additional information on OdinText Contextual Sentiment/Noise Reduction]

Attensity Clarabridge Text Analytics Softwaer Comparison

The real innovation in word processing was not the technology, but its impact: Word processing simplified and democratized publishing.

OdinText doesn’t require a team of linguists, data scientists or expert consultants to set up before you can use it or reuse it. OdinText enables anyone in your organization to quickly, easily conduct sophisticated analyses of any unstructured text data—survey open-ends, call center transcripts, email, social media, discussion boards—to deliver immediate insights.


In short, OdinText is built for the Analyst in mind - faster setup, cleaner data, better insight, all within a simple interface everyone -- especially Analysts -- can use.

Find out for yourself. Contact us for a demo.


Yours fondly, @TomHCAnderson


[NOTE: Tom is Founder and CEO of OdinText Inc.. A long time champion of text mining, in 2005 he founded Anderson Analytics LLC, the first consumer insights/marketing research consultancy focused on text analytics. He is a frequent speaker and data science guest lecturer at university and research industry events.]

Text Analytics Tips

Text Analytics Tips, with your Hosts Tom & Gosia: Introductory Post Today, we’re blogging to let you know about a new series of posts starting in January 2016 called ‘Text Analytics Tips’. This will be an ongoing series and our main goal is to help marketers understand text analytics better.

We realize Text Analytics is a subject with incredibly high awareness, yet sadly also a subject with many misconceptions.

The first generation of text analytics vendors over hyped the importance of sentiment as a tool, as well as ‘social media’ as a data source, often preferring to use the even vaguer term ‘Big Data’ (usually just referring to tweets). They offered no evidence of the value of either, and have usually ignored the much richer techniques and sources of data for text analysis. Little to no information or training is offered on how to actually gain useful insights via text analytics.

What are some of the biggest misconceptions in text analytics?

  1. “Text Analytics is Qualitative Research”

FALSE – Text Analytics IS NOT qualitative. Text Analytics = Text Mining = Data Mining = Pattern Recognition = Math/Stats/Quant Research

  1. It’s Automatic (artificial intelligence), you just press a button and look at the report / wordcloud

FALSE – Text Analytics is a powerful technique made possible thanks to tremendous processing power. It can be easy if using the right tool, but just like any other powerful analytical tools, it is limited by the quality of your data and the resourcefulness and skill of the analyst.

  1. Text Analytics is a Luxury (i.e. structured data analysis is of primary importance and unstructured data is an extra)

FALSE – Nothing could be further from the truth. In our experience, usually when there is text data available, it almost always outperforms standard available quant data in terms of explaining and/or predicting the outcome of interest!

There are several other text analytics misconceptions of course and we hope to cover many of them as well.

While various OdinText employees and clients may be posting in the ‘Text Analytics Tips’ series over time, Senior Data Scientist, Gosia, and our Founder, Tom, have volunteered to post on a more regular basis…well, not so much volunteered as drawing the shortest straw (our developers made it clear that “Engineers don’t do blog posts!”).

Kidding aside, we really value education at OdinText, and it is our goal to make sure OdinText users become proficient in text analytics.

Though Text Analytics, and OdinText in particular, are very powerful tools, we will aim to keep these posts light, fun yet interesting and insightful. If you’ve just started using OdinText or are interested in applied text analytics in general, these posts are certainly a good start for you.

During this long running series we’ll be posting tips, interviews, and various fun short analysis. Please come back in January for our first post which will deal with analysis of a very simple unstructured survey question.

Of course, if you’re interested in more info on OdinText, no need to wait, just fill out our short Request Info form.

Happy New Year!

Your friends @OdinText

Text Analytiics Tips T G

[NOTE: Tom is Founder and CEO of OdinText Inc.. A long time champion of text mining, in 2005 he founded Anderson Analytics LLC, the first consumer insights/marketing research consultancy focused on text analytics. He is a frequent speaker and data science guest lecturer at university and research industry events.

Gosia is a Senior Data Scientist at OdinText Inc.. A PhD. with extensive experience in content analytics, especially psychological content analysis (i.e. sentiment analysis and emotion in text), as well as predictive analytics using unstructured data, she is fluent in German, Polish and Spanish.]


Forbes: Text Analytics Gurus Debunk Four Big Data Myths

Four Big Data and Text Analytics Myths Debunked [Visit for more detailed article]

There are a lot of myths out there surrounding next generation research techniques around data from small to big. Text analytics, if done correctly, can offer insights into problems from a new more encompassing and accurate perspective.

OdinText CEO, Tom H. C. Anderson, and CTO Chris Lehew recently spoke to Forbes Magazine about some of the common analytics and text analytics myths they frequently encounter, and gave real examples of how OdinText users are benefiting from superior insights provided by the Next Generation Text AnalyticsTM platform.

The four data myths covered today in Forbes were:

Myth 1: Big Data Survey Scores Reign Supreme

 A myth propagated by Bain Consulting, without any evidence to justify the claims, is that a survey metric asking 'likeliness to recommend' is related to business growth. OdinText found absolutely no link between survey metrics like NPS and revenue when analyzing 800,000 customer surveys for Shell Oil/Jiffy Lube.

Actual customer text comments proved to be the single best predictor of real behavior from loyalty/returns to revenue.


Myth 2: Bigger Social Media Data Analysis Is Better


Many believe big data must have value simply because it's big - but it’s more about smart data than big data. Using OdinText Coca-Cola’s Social Media Hub found that only smarter text analytics and data translate to better insights.


Myth 3: New Data Sources Are The Most Valuable

 Many companies go looking for new data on social sites and elsewhere while failing to first look too what valuable data they may already be collecting . Campbell’s Soup for instance realized that they could get better insights by using OdinText to listen to the quarter of a million customers that contact their customer service department each year than relying on anything found on Twitter.


Myth 4: Keep Your Eye On The Ball By Focusing On How Customers View You

 Sometimes online discussion boards and other consumer generated media is the best place to find insights. But companies often focus just on what is said about them. Starwood hotels among others have realized the benefit of doing comparative analysis VS. their competition, arguably the single best use of this type of data.


You can read more detail about the four data myths discussed above in Forbes Magazine article here.

Please visit our blog in the coming months as we roll out a new series of “Text Analytics Tips” with experts from OdinText discussing more myths and sharing tips and techniques on how to drive better more actionable insights through smarter Next Generation Text AnalyticsTM

Coca-Cola Social Media Team Chooses OdinText Text Analytics Software and Wins 2015 ARF Award

Coca-Cola Leverages OdinText analytics to develop new approach to social listening and wins ARF’s 2015 RE:Think Your Future “Make Your Mark” Award!

At the Advertising Research Foundation’s (ARF) annual RE:Think Conference this week?

Please join OdinText and The Coca-Cola Company this coming week at the 2015 Advertising Research Foundation’s annual RE:THINK event to learn more about the future of big data and text analytics.


On Monday in SuttTomCokeTextAnalyticson South, 2nd Floor at 3:30, OdinText Founder and CEO Tom H. C. Anderson will be joined by several other industry leaders in what promises to be a very interesting panel entitled “Applying Mobile Big Data at the Intersection of Content, Context & User Analytics”. As unstructured data is the key to the future of mobile, text analytics promises to be a key part of the discussion.



On Wednesday afternoon in the Grand Ballroom, The Coca-Cola Company’s Digital Anthropologist, Allison Barnes and Global Media Insights Director, Justin De Graaf, are being honored for their very interesting work and award winning paper entitled “Digital Anthropology: Researching Audiences Online”.



Allison and Justin will briefly discuss how Coca-Cola is leveraging OdinText to understand specific consumer groups, developing an innovative new approach that works across the Coca-Cola system. An approach that is scalable, utilizes existing tools, works across all markets and adapts to custom requirements.

“The new technique moves beyond word clouds [and other first generation social listening features] which represent the old way, and towards true analytics, which we’re getting from OdinText”

As Allison points out in their paper, great text analytics is critical for The Coca-Cola Company since “Social is a new avenue our consumers use to share their lives, and tapping in to those shared thoughts allows us to become more closely tied to the things that matter most to them. Our ability to continue innovating communications, products and happiness is at the heart of our future and harnessing social allows us to do exactly that”.

Congratulations to Justin, Allison and the rest of the social media team at Coca-Cola for a job well done!

We look forward to seeing you at The ARF Monday or Wednesday. And as always we welcome your questions about OdinText and text analytics anytime!

New to Text Analytics?

6 lessons for CIOs and CMOs who are new to text analytics(Free White Paper)

Unstructured data is the most prevalent form of information on the planet. It also underpins much of our communication. It exists in our e-mails, surveys, social media accounts, call center logs, etc. With a strong text analytics strategy in place, companies can get critical information from this data to drive better business decisions.

You may recall my blog interviews with several client side analytics managers ahead of this years’ Useful Business Analytics Summit.

Data Driven Business has compiled a free white paper which focuses on the business benefits (and challenges!) of text analytics from the perspectives of 4 of these experts from Highmark Health, Toyota, Mozilla, and Visa (All who are slated to speak at the 13th Annual Text Analytics Summit West on November 4-5 in San Francisco.

You can download the free whitepaper here:

Some of the issues covered include:

· 6 lessons for CIOs and CMOs who are new to text analytics

· Identifying KPIs for your text analytics initiatives

· Barriers to adoption and how to overcome them

Big thanks to the following contributors:

Mark Pitts, Vice President, Enterprise Informatics, Data & Analytics at Highmark Health

Farouk Ferchichi, Executive Director at Toyota Financial Services

Matthew P.T. Ruttley, Manager of Data Science at Mozilla Corporation

Ramkumar Ravichandran, Director, Analytics at Visa


[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of patented Next Generation Text Analytics™software platform OdinText. For more information and to inquire about software licensing visit ODINTEXT INFO REQUEST]

Mr Big Data VS. Mr Text Analytics

[Interview re-posted w/ permission from Text Analytics News]

Mr. Big Data & Mr. Text Analytics Weigh In Structured VS. Unstructured Big Data


kirk_borne Text Analytics News

If you pay attention to Big Data news you’re sure to have heard of Kirk Borne who’s well respected views on the changing landscape are often shared on social media. Kirk is professor of Astrophysics and Computational Science at George Mason University. He has published over 200 articles and given over 200 invited talks at conferences and universities worldwide. He serves on several national and international advisory boards and journal editorial boards related to big data



tom_anderson Text Analytics News

Tom H. C. Anderson was an early champion of applied text analytics, and gives over 20 conference talks on the topic each year, as well as lectures at Columbia Business School and other universities. In 2007 he founded the Next Gen Market Research community online where over 20,000 researchers frequently share their experiences online. Tom is founder of Anderson Analytics, developers of text analytics software as a service OdinText. He serves on the American Marketing Association’s Insights Council and was the first proponent of natural language processing in the marketing research/consumer insights field.


Ahead of the Text Analytics Summit West 2014, Data Driven Business caught up with them to gain perspectives on just how important and interlinked Big Data is with Text Analytics.


Q1. What was the biggest hurdle that you had to overcome in order to reach your current level of achievement with Big Data Analytics?

KB: The biggest hurdle for me has consistently been cultural -- i.e., convincing others in the organization that big data analytics is not "business as usual", that the opportunities and potential for new discoveries, new insights, new products, and new ways of engaging our stakeholders (whether in business, or education, or government) through big data analytics are now enormous.

After I accepted the fact that the most likely way for people to change their viewpoint is for them to observe demonstrated proof of these big claims, I decided to focus less on trying to sell the idea and focus more on reaching my own goals and achievements with big data analytics. After making that decision, I never looked back -- whatever successes that I have achieved, they are now influencing and changing people, and I am no longer waiting for the culture to change.

THCA: There are technical/tactical hurdles, and methodological ones. The technical scale/speed ones were relatively easy to deal with once we started building our own software OdinText. Computing power continues to increase, and the rest is really about optimizing code.

The methodological hurdles are far more challenging. It’s relatively easy to look at what others have done, or even to come up with new ideas. But you do have to be willing to experiment, and more than just willingness, you need to have the time and the data to do it! There is a lot of software coming out of academia now. They like to mention their institution in every other sentence “MIT this” or ‘UCLA that”. The problem they face is twofold. On the one hand they don’t have access to enough real data to see if their theories play out. Secondly, they don’t have the real world business experience and access to clients to know what things are actually useful and which are just novelty.

So, our biggest hurdle has been the time and effort invested through empirical testing. It hasn’t always been easy, but it’s put me and my company in an incredibly unique position.

Q2. Size of data, does it really matter? How much data is too little or too much?

THCA: Great question, with text analytics size really does matter. While it’s technically possible to get insights from very small data, for instance on our blog during the elections one of my colleagues did a little analysis of Romney VS. Obama debate transcripts, text analytics really is data mining, and when you’re looking for patterns in text, the more data you have the more interesting relationships you can find.

KB: Size of data doesn't really matter if you are just getting started. You should get busy with analytics regardless of how little data you have. The important thing is to identify what you need (new skills, technologies, processes, and data-oriented business objectives) in order to take advantage of your digital resources and data streams. As you become increasingly comfortable with those, then you will grow in confidence to step up your game with bigger data sets. If you are already confident and ready-to-go, then go! The big data revolution is like a hyper-speed train -- you cannot wait for it to stop in order to get on board -- it isn't stopping or slowing down! At the other extreme, we do have to wonder if there is such a thing as too much data. The answer to this question is "yes" if we dive into big data's deep waters blindly without the appropriate "swimming instruction" (i.e., without the appropriate skills, technologies, processes, and data-oriented business objectives). However, with the right preparations, we can take advantage of the fact that bigger data collections enable a greater depth of discovery, insight, and data-driven decision support than ever before imagined.

Q3. What is the one thing that motivates and inspires you the most in your Big Data Analytics work?

KB: Discovery! As a scientist, I was born curious. I am motivated and inspired to ask questions, to seek answers, to contemplate what it all means, and then to ask more questions. The rewards from these labors are the discoveries that are made along the way. In data analytics, the discoveries may be represented by a surprising unexpected pattern, trend, association, correlation, event, or outlier in the data set. That discovery then becomes an intellectual challenge (that I love): What does it mean? What new understanding does this discovery reveal about the domain of study (whether it is astrophysics, or retail business, or national security, or healthcare, or climate, or social, or whatever)? The discovery and the corresponding understanding are the benefits of all the hard work of data wrangling.

THCA: Anyone working with analytics has to be curious by nature. Satisfying that curiosity is what drives us. More specifically in my case, if our clients get excited about using our software and the insights they’ve uncovered, then that really gets me and my whole team excited. This can be challenging, and not all data is created equal.

It can be hard to tell someone who is excited about trying Text Analytics that their data really isn’t suitable. The opposite is even more frustrating though, knowing that a client has some really interesting data but is apprehensive about trying something new because they have some old tools lying around that they haven’t used, or because they have a difficult time getting access to the data because it’s technically “owned” by some other department that doesn’t ‘Get’ analytics. But helping them build a case and then helping them look good by making data useful to the organization really feeds into that basic curiosity. We often discover problems to solve we had no idea existed. And that’s very inspiring and rewarding.

Q4. Which big data analytics myth would you like to squash right here and now?

KB: Big data is not about data volume! That is the biggest myth and red herring in the business of big data analytics. Some people say that "we have always had big data", referring to the fact that each new generation has more data than the previous generation's tools and technologies are able to handle. By this reasoning, even the ancient Romans had big data, following their first census of the known world. But that's crazy. The truth of big data analytics is that we are now studying, measuring, tracking, and analyzing just about everything through digital signals (whether it is social media, or surveillance, or satellites, or drones, or scientific instruments, or web logs, or machine logs, or whatever). Big data really is "everything, quantified and tracked". This reality is producing enormously huge data volumes, but the real power of big data analytics is in "whole population analysis", signaling a new era in analytics: the "end of demographics", the diminished use of small samples, the "segment of one", and a new era of personalization. We have moved beyond mere descriptive analysis, to predictive, prescriptive, and cognitive analytics.

THCA: Tough one. There are quite a few. I’ll avoid picking on “social media listening” for a bit, and pick something else. One of the myths out there is that you have to be some sort of know it all ‘data scientist’ to leverage big data. This is no longer the truth. Along with this you have a lot dropping of buzz words like “natural language processing” or “machine learning” which really don’t mean anything at all.

If you understand smaller data analytics, then there really is no reason at all that you shouldn’t understand big data analytics. Don’t ever let someone use some buzz word that you’re not sure of to impress you. If they can’t explain to you in layman’s terms exactly how a certain software works or how exactly an analysis is done and what the real business benefit is, then you can be pretty sure they don’t actually have the experience you’re looking for and are trying to hide this fact.

Q5.What’s more important/valuable, structured or unstructured data?

KB: Someone said recently that there is no such thing as unstructured data. Even binary-encoded images or videos are structured. Even free text and sentences (like this one) are structured (through the rules of language and grammar). Even some meaning this sentence has. One could say that analytics is the process of extracting order, meaning, and understanding from data. That process is made easier when the data are organized into databases (tables with rows and columns), but the importance and value of the data are inherently no more or no less for structured or unstructured data. Despite these comments, I should say that the world is increasingly generating and collecting more "unstructured data" (text, voice, video, audio) than "structured data" (data stored in database tables). So, in that sense, "unstructured data" is more important and valuable, simply because it provides a greater signal on the pulse of the world. But I now return to my initial point: to derive the most value from these data sources, they need to be analyzed and mined for the patterns, trends, associations, correlations, events, and outliers that they contain. In performing that analysis, we are converting the inherent knowledge encoded in the data from a "byte format" to a "structured information format". At that point, all data really become structured.

THCA: A trick question. We all begin with a question and relatively unstructured data. The goal of text analytics is structuring that data which is often most unstructured.

That said, based on the data we often look at (voice of customer surveys, call center and email data, various other web based data), I’ve personally seen that the unstructured text data is usually far richer. I say that because we can usually take that unstructured data and accurately predict/calculate any of the available structured data metrics from it. On the other hand, the unstructured data usually contain a lot of additional information not previously available in the structured data. So unlocking this richer unstructured data allows us to understand systems and processes much better than before and allows us to build far more accurate models.

So yes, unstructured/text data is more valuable, sorry.

Q6. What do you think is the biggest difference between big data analysis being done in academia vs in business?

KB: Perhaps the biggest difference is that data analysis in academia is focused on design (research), while business is focused on development (applications). In academia, we are designing (and testing) the optimal algorithm, the most effective technique, the most efficient methodology, and the most novel idea. In business, you might be 100% satisfied to apply all of those academic results to your business objectives, to develop products and services, without trying to come up with a new theory or algorithm. Nevertheless, I am actually seeing more and more convergence (though that might be because I am personally engaged in both places through my academic and consulting activities). I see convergence in the sense that I see businesses who are willing to investigate, design, and test new ideas and approaches (those projects are often led by data scientists), and I see academics who are willing to apply their ideas in the marketplace (as evidenced by the large number of big data analytics startups with university professors in data science leadership positions). The data "scientist" job category should imply that some research, discovery, design, modeling, and hypothesis generation and testing are part of that person's duties and responsibilities. Of course, in business, the data science project must also address a business objective that serves the business needs (revenue, sales, customer engagement, etc.), whereas in academia the objective is often a research paper, or a conference presentation, or an educational experience. Despite those distinctions, data scientists on both sides of the academia-business boundary are now performing similar big data analyses and investigations. Boundary crossing is the new normal, and that's a very good thing.

THCA: I kind of answered that in the first question. I think academics have the freedom and time to pursue a research objective even if it doesn’t have an important real outcome. So they can pick something fun, that may or may not be very useful, such as are people happier on Tuesdays or Wednesday’s? They’ll often try to solve these stated objectives in some clever ways (hopefully), though there’s a lot of “Pop” research going on even in academia these days. They are also often limited in the data available to them, having to work with just a single data set that has somehow become available to them.

So, academia is different in that they raise some interesting fun questions, and sometimes the ideas borne out of their research can be applied to business.

Professional researchers have to prove an ROI in terms of time and money. Of course, technically we also have access to both more time and more money, and also a lot more data. So an academic team of researcher working on text analytics for 2-3 years is not going to be exposed to nearly as much data as a professional team.

That’s also why academic researchers often seem so in love with their models and accuracy. If you only have a single data set to work with, then you split it in half and use that for validation. In business on the other hand, if you are working across industries like we do, while we certainly may build and validate models for a specific client, we know that having a model that works across companies or industries is nearly impossible. But when we do find something that works, you can bet it’s going to be more likely to be useful.

Battle of the Methodologies

Text Analytics VS. Social Media Monitoring VS. MROC’s VS. Mobile GreatMethodologyDebateTextAnalyticsMROCsMobileSocialMediaMonitoring

Many of you are probably already planning on attending IIEX (Insight Innovation Exchange) in Atlanta June 16th. I’ve spoken at their previous events and talked to many attendees who liked the excitement and ability to explore many of the new techniques vying for attention in the consumer insights space.

That’s why this year I was very excited to be asked to take part in a very special panel, one that I understand will be a Battle of the Methodologies!


Let me explain. The folks over at Greenbook identified the five 'Next Gen' Research techniques which have been most disruptive to the status quo. Some like neuromarketing and social media monitoring are relatively new and unproven, others like Mobile and and text analytics arguably have been around for quite some time. However all have now reached mainstream and are being considered by research directors globally.

A key leader in each of these four areas was then selected and asked if they would like to participate in a debate *panel* explaining why their research technique/methodology is more important than any of the others – the method to rule them all!

OK, I know, in fact I think most of us know that no methodology/technique is the correct approach 100% of the time, and that problem and data identification should always come before selecting the proper method. However, that doesn’t mean that all techniques are equally proven, useful, efficient and yes Important.

That is up for debate, and it’s a debate worth having. Therefore I’m very honored and excited to participate in this methodology brawl with my esteemed colleagues, each of whom is a pioneer in their own respective discipline. Taking part will be:


Representing Text Analytics

Tomtom[Yours truly] Tom H. C. Anderson, Founder Anderson Analytics – OdinText a Next Generation Text Analytics solution






Representing Neuro Marketing

steve_gencoSteve Genco, Managing Partner, Intuitive Consumer Insights and lead author, Neuromarketing for Dummies (Wiley, 2013)







Representing Social Media Monitoring


Michalis A. Michael, CEO at DigitalMR, which is a digital market research consultancy with proprietary platforms for social media listening and private online communities






Representing Mobile

MarkMark Michelson, CEO Threads Strategic Research & Consulting, and Executive Director MMRA (Mobile Marketing Research Association)







Representing Online Communities (MROC's)

NielsNiels Schillewaert, PhD. Managing Partner and Founder InSites Consulting a new generation agency stretching boundaries of marketing research. Helping global brands become locally relevant







Additionally the panel will be moderated by none other than Eileen Campbell, Chief Marketing Officer IMAX (Previously CEO of Millward Brown - WPP agency group)

Certainly there is some overlap in expertise and offering among some of the panelists. That said, it’s bound to be a bloody battle -may the best Methodology Win!

If you’ll be attending the event please let me know. I’ll need all the support I can get ;)




[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of patented Next Generation Text Analytics™ software platform OdinText. For more information and to inquire about software licensing visit ODINTEXT INFO REQUEST]

[Above also posted on the Next Gen Market Research blog]

Why I FourSquare – Personal Big Data and Serendipity!

FourSquare I’ve been asked more than once, even by some of my more social media savvy Facebook friends why I use Foursquare. “What’s the point?!” they say.

Well other than the more obvious benefits which include receiving the occasional promotional discounts I also occasionally do it as part of my personal Big Data diligence. At some point, a couple of years from now (probably sooner) I believe we’ll all have tremendous amounts of personal big data, and those of us who have more of it will be able to do more interesting analysis and comparisons. As our software ( thanks to text analytics), can more easily handle this unstructured data and connect it to other relevant information I believe it may well enrich not just my ability to tap into memories, but even help with more advanced analysis related to my health and wellbeing.

But that’s not all, it can also be an integral part of networking and can provide that serendipitous aspect that is missing from most of the mainstream social media networks.

Take this past week for instance when I made an impromptu trip to Toronto Canada. Prior to arriving on Friday I posted a question to my Facebook friends about recommendations for restaurants etc. In a matter of minutes I had several suggestions from Canadian friends and others who had been to the city.

On Saturday my first restaurant choice was totally booked, and so I checked both Yelp and FourSquare for nearby options. After finding a suitable sushi place, and having enjoyed a great meal, I realized I hadn’t checked in and so I did. Within less than 5 minutes I saw familiar face I hadn’t seen in quite some time. The President of the Canadian Marketing Research Association (MRIA) had been walking by the restaurant, saw my check-in and came in to say hello. I invited her to join us and we shared a beer and some Sake, while catching up on everything from research industry gossip to US-Canadian politics. She also gave me some great tips on what to do the next day. I always find travel more rewarding when you can get the local insights into life, including important do’s and don’ts.

Believe it or not, this has happened to me more than once - and that’s why I use FourSquare.

Thanks Sandy! ;)




[Full Disclosure: Tom H. C. Anderson is Managing Partner of Anderson Analytics, developers of a patented Next Generation approach to text analytics known as OdinText. For more information and to inquire about software licensing visit OdinText INFO Request.]