Posts tagged Insights Associaiton
What you Need to Know Before Buying AI/Machine Learning

7 Things to Know About AI/Machine Learning (Boiled Down to two Cliff Notes that are even more important).

In case you missed our session on Artificial Intelligence and Machine Learning (AI/ML) at the Insights Associations’ NEXT conference last week, I thought I would share a bit on the blog about what you missed. We had a full room, with some great questions both during and after the session. However, 30 minutes wasn’t enough time to cover everything thoroughly. In the end we agreed on four takeaways:

  • AI is part of how research & insights pros will address the ever-increasing demand for fast research results
  • AI Helps focus on the most important data
  • AI can’t compensate for bad data
  • AI isn’t perfect

So today I thought I would share seven additional points about AI/ML that I often get questions on, and then at the end of this post I’m going to share the ‘Cliff Notes’, i.e. I’m going to share just the 2 most important things you really need to know.  So, unless you want to geek out with me a bit, feel free to scroll to the bottom.

OK, first, before we can talk about anything, we need to define what Artificial Intelligence (AI) is and isn’t.

1. AI/ML definition is somewhat fuzzy

AI, and more specifically machine learning (ML) is a term that is abused almost as often as it is used. On the one hand this is because a lot of folks are inaccurately claiming to be using it, but also because not unlike big data, its definitions can be a bit unclear, and don’t always make perfect sense.

Let’s take this common 3-part regression analysis process:

  1. Data Prep (pre-processing including cleaning, feature identification, and dimension reduction)
  2. Regression
  3. Analysis of process & reporting

This process, even if automated would not be considered machine learning. However, switch out regression with a machine learning technique like Neural Nets, SVM, Decision Trees or Random Forests and bang, it’s machine learning. Why?

Regression models are also created to predict something, and they also require training data. If the data is linear, then there is no way any of these other models will beat regression in terms of ROI. So why would regression not be considered machine learning?

Who knows. Probably just because the first writers of the first few academic papers on ML refenced these techniques and not regression as ML. It really doesn’t make much sense.

2. There are basically 2 types of ML

Some ML approaches are binary like SVM (Support Vector Machines), for predicting something like male or female, and others like Decision Trees are multi class classification.

If you are using decision trees to predict an NPS rating on an 11 point scale then that’s a multi class problem. However, you can ‘trick’ binary techniques like SVM to solve the multi class problem by setting it up to run multiple times.

Either way, you are predicting something.

3. ML can be slow

Depending on the approach used, like Neural Nets for instance, training a model can take several days on a normal computer. There are other issues with Neural Nets as well, like the difficulty for humans to understand and control what they are doing.

But let’s focus on speed for now. Of course, if you can apply a previously trained model on very similar data, then results will be very fast indeed. This isn’t always possible though

If your goal is to insert ML into a process to solve a problem which a user is waiting for, then training an algorithm might not be a very good solution. If another technique, ‘machine learning’ or not, can solve the problem much faster with similar accuracy, then that should be the approach to use.

4. Neural Nets are not like the brain

I’ll pick on Neural Nets a bit more, because they are almost a buzz word unto themselves. That’s because a lot of people have claimed they work like the human brain. This isn’t true. If we’re going to be honest, we’re not sure how the human brain works. In fact, what we do know about the human brain makes me think the human brain is quite different.

The human brain contains nearly 90 billion neurons, each with thousands of synapses. Some of these fire and send information for a given task, some will not fire, and yet others fire and do not send any information. The fact is we don’t know exactly why. This is something we are still working on with hopes that new more powerful quantum computers may give us some insight.

We can however map some functions of the brain to robotics to do things like lift arms, without knowing exactly what happens in between.

There is one problematic similarity between the brain and Neural Nets though. That is, we’re not quite sure how Neural Nets work either. When running a Neural Net, we cannot easily control or explain what happens in the intermediary nodes. So, this (along with speed I mentioned earlier) is more of a reason to be cautious about using Neural Nets.

5. Not All Problems are best solved with Machine Learning

Are all problems best solved with ML? No, probably not.

Take pricing as an example. People have solved for this problem for years, and there are many different solutions depending on your unique situation. These solutions can factor in everything from supply and demand, to cost.

Introducing machine learning, or even just a simpler non-ML based automated technique can sometimes cause unexpected problems. As an example, consider the automated real-time pricing model which Uber used to model supply and demand as inputs. When fares skyrocketed to over $1,000 as drunk people were looking for a ride on New Years eve, the model created a lot of angry customers and bad press.

More on dangers of AI/ML in a bit…

6. It’s harder to beat humans than you think

One of the reasons ML is often touted as a solution is because of how much better than humans computers allegedly are. While theoretically there is truth to this, when applied to real world situations we often see a less ideal picture.

Take self driving cars as an example. Until recently they were touted as “safer than humans”. That was until they began crashing and blowing up.

Take the recent Tesla crash as an example. The AI/ML accidentally latched onto an older faded lane line rather than the newly painted correct lane line and proceeded without breaking, at full speed, into a head on collision with a divider. A specific fatal mistake no human would have been likely to make.

The truth is if we remove driving under the influence and falling asleep from the statistics (two things that are illegal anyway), then human accident statistics are incredibly low.

7. ML is Context Specific!

This is an important one. IBM Watson might be able to Google Lady Gaga’s age quickly, but Watson will be completely useless in identifying her in a picture. Machine learning solutions are extremely context specific.

This context specificity also comes into play when training any type of model. The model will only be as good as the training data used to create it, and the similarity to future data it is uses for predictions.

Model validation methods only test the accuracy of the model on the same exact type of data (typically a random portion of the same data), it does not test the quality of the data itself, nor the application of this model on future data other than the training data.

Be wary of anyone who claims their AI does all sorts of things well, or does it with extremely 100% accuracy.

My final point about Machine Learning & two Cliff Notes…

If some of the above points make it sound as if I’m not bullish on machine learning, I want to clarify that in fact I am. At OdinText we are continuously testing and implementing ML when it makes sense. I’m confident that we as an industry will get better and better at machine learning.

In the case of Tesla above, there are numerous ways to make the computers more efficient, including using special paint that would be easier for computer cameras to see, and traffic lights that send signals telling the computer stating “I am red”, “I am Green” etc., rather than having to guess it via color/light sensing. Things will certainly change, and AI/ML will play an important part.

However, immediately after my talk at the Insights Association I had two very interesting conversations on how to “identify the right AI solution”? In both instances, the buyer was evaluating vendors that made a lot of claims. Way too many in my opinion.

If you forget everything else from today’s post, please remember these two simple Cliff Notes on AI:

  1. You Don’t Buy AI, you buy a solution that does a good job solving your need (which may or may not involve AI)
  2. Remember AI is context specific, and not perfect. Stay away from anyone who says anything else. Select vendors you know you can trust.

There’s no way to know whether something is AI or not without looking at the code.

Unlike academics who share everything under peer review, companies protect their IP, Trade Secrets and code, so there will technically be no way for you to evaluate whether something actually is “AI” or not.

However, the good news is, this makes your job easier. Rather than reviewing someone’s code your job is simply still to decide whether the products solves your needs well or not.

In fact, in my opinion it is far more important to choose a vendor who is honest with you about what they can do to solve your problems. If a vendor claims they have AI everywhere that solves all kinds of various needs, and does so with 100% accuracy, run!

@TomHCAnderson

AI and Machine Learning NEXT at The Insights Association
Insight practitioners from Aon, Conagra and Verizon speak out on what they think about AI and Machine Learning

Artificial Intelligence and Machine Learning are hot topics today in many fields, and marketing research is no  exception. At the Insights Association’s NEXT conference on May 1 in NYC I've been asked to take part in a practitioner panel on AI to share a bit about how we are using AI in natural language processing and analytics at OdinText.

While AI is an important part of what data mining and text analytics software providers like OdinText do, before the conference I thought I’d reach out to a couple of the client-side colleagues to see what they think about the subject.

With me today I have David Lo, Associate Partner at the Scorpio Partnership (a collaboration between McLagan and the Aon Hewitt Corporation) Thatcher Schulte, Sr. Director, Strategic Insights at Conagra Brands, and Jonathan Schwedel, Consumer & Marketplace Insights at Verizon, all who will also be speaking at NEXT.

THCA: Artificial Intelligence means different things to different people and companies. What does it mean to you, and how if at all you are planning to use it in your departments?

Thatcher Schulte – Conagra:

Artificial intelligence is like many concepts we discuss in business, it’s a catch all that loses its meaning as more and more people use it.  I’ve even heard people refer to “Macros” as AI.  To me it means trying to make machines make decisions like people would, but that would beg the question on whether it would be “intelligent.”  I make stupid decisions all the time.

We’re working with Voice to make inferences on what help consumers might need as they make decisions around food.

Jonathan Schwedel – Verizon:

I'm not a consumer insight professional - I'm a data analyst who works in the insights department, so my perspective is different. There are teams in other parts of Verizon who are doing a lot with more standard artificial intelligence and machine learning approaches, so I want to be careful not to conflate the term with broader advanced analytics. I have this image of cognitive scientists sitting in a lab, and am tempted to reduce "AI" to that.

For our specific insights efforts, we work on initiatives that are AI-adjacent - with automation, predictive modeling, machine learning, and natural language processing, but with a few exceptions those efforts are not scaled up, and are ad hoc on a project by project basis. We dabble with a lot of the techniques that are highlighted at NEXT, but I'm not knowledgeable enough about our day to day custom research efforts to speak well to them. One of the selling points of the knowledge management system we are launching is that it's supposed to leverage machine learning to push the most relevant content to our researchers and partners around our company.

David Lo – Scorpio Partnership/McLagan:

Working in the financial services space and specifically within wealth management, AI is a hot topic as it relates to how it will change advice delivery

[we are looking at using it for] Customer journey mapping through the various touchpoints they have with an organization.

 

THCA: There’s a lot of hype these days around AI. What is your impression on what you’ve been hearing, and about the companies you’ve been hearing it from, is it believable?

Thatcher Schulte - Conagra:

I don’t get pitched on AI a lot except through email, which frankly hurts the purpose of those people pitching me solutions.  I don’t read emails from vendors.

Jonathan Schwedel – Verizon:

It's easy to tell if someone does not have a minimum level of domain expertise. The idea that any tool or platform can provide instant shortcuts is fiction. Most of the value in these techniques are very matter of fact and practical. Fantastic claims demand a higher level of scrutiny. If instead the conversation is about how much faster, cheaper, or easier they are, those are at least claims that can be quickly evaluated.

David Lo – Scorpio Partnership/McLagan:

Definitely a lot of hype.  I think as it relates to efficiency, the hype is real.  We will continue to see complex tasks such as trade execution optimized through AI.

 

THCA: For the Insights function specifically, how ready do you think the idea of completely unsupervised vs. supervised/guided AI is? In other words, do you think that the one size fits all AI provided by likes of Microsoft, Amazon, Google and IBM are very useful for research, or does AI need to be more customized and fine tuned/guided before it can be very useful to you?

And related to this, what areas of Market Research do you thing AI currently is better suited to AI?

 Thatcher Schulte - Conagra:

Data sets are more important to me than the solutions that are in the market.  Food decision making is specialized and complex and it varies greatly by what life stage you are in and where you live. Valid data around those factors are frankly more important than the company we push the data through.

David Lo – Scorpio Partnership/McLagan:

Guard rails are always important, particularly as it relates to unique customer needs.

[In terms of usefulness to market research], Data mining

Jonathan Schwedel – Verizon:

Most custom quantitative research studies use small sample sizes, making it often not feasible to do bespoke advanced analytics. When you are working with much larger data sets (the kind you'd see in analytics as a function as opposed to insights), AWS and Azure let you scale, especially with limited resources. It's a good general approach to use algorithmic type approaches with brand new data sets, and then start customizing when you hit the point of diminishing returns, in a way that your work can later be automated at scale.

[In regard to marketing research] It depends how you're defining research - are we broadening that to customer experience? Then text analytics is a most prominent area, because there are many prominent use cases for large companies at the enterprise level. If "market research" covers broader buckets of customer data, then there's potentially a lot you can do.

 

THCA: OK, so which areas are currently less well suited to AI?

David Lo – Scorpio Partnership/McLagan:

Hard to say, but probably less suited toward qualitative research.  In my line of business we do a lot of work among UHNW investors where sample sizes are very small and there isn’t a lot of activity in the online space.

Jonathan Schwedel – Verizon:

I think sample size is often an issue when talking about research studies. Then it comes down to the research design. Is the machine learning component going to be baked in from the start, or is it just bolted on? A lot of these efforts are difficult to quantify. Verizon's insights group learns things all the time from talking to and observing consumers that we would not have otherwise thought to ask.

 

THCA: Does anyone have thoughts on usefulness of chat bots and/or other social media/twitter bots currently?

Jonathan Schwedel – Verizon:

They could potentially allow you to collect a lot more data, and reach under-represented consumers groups in the channels that they want to be in. A lot of our team's focus at Verizon is on the user experience and building a great digital experience for our customers. I think they will be important tools to understand and improve in that area.

 

THCA: Realistically where do you see AI in market research being 3-4 years from now?

David Lo – Scorpio Partnership/McLagan:

Integrated more fully with traditional quantitative research techniques, with researchers re-focusing their efforts on the more creative and thoughtful interpretations of the output.

Jonathan Schwedel – Verizon:

They will provide some new techniques that will be important for specific use cases, but I think the bulk of the fruitful efforts will come from automation and improved scalability. The desire to do more with less is pretty universal, and there's a good roadmap there. The prospect of genuinely groundbreaking insights offers a lot more uncertainty, but it would be great if we do see that level of innovation.

 

Big thanks to Jonathan, David and Thatcher for sharing their insights and opinions on AI.

If you’re interested in further discussion on AI and Machine Learning please feel free too post a comment here, or join me for the 'What’s New & What’s Ahead for AI & Machine Learning?' Panel on May 1st . I will be joined by John Colias of Decision Analyst, Andrew Konya of remesh, and moderator Kathryn Korostoff of Research Rockstar.

-Tom H. C. Anderson @OdinText

 

PS. If you would like to learn more about how OdinText can help you better understand your customers and employees feel free to request more info here. If you’re planning on attending the confernece feel free use my speaker code for a $150 discount [ODINTEXT]. I look forward to seeing some of you at the event!

 

Attending Insights Associations' NEXT 2017 Tomorrow?

Are you attending The Insights Associations’ Inaugural Analytics event NEXT in NYC tomorrow? If so please stop by the OdinText presentation on text analytics at 3:30. I’ve been asked to present not just some of the findings of a very exciting blog post, but also explain how you too can conduct this type of advanced analysis quickly and affordably.

The session is even more exciting than the title, called 'Tap the Power of a Single Open-Ended Question' (follow the link for a more detailed description).

I hope to see you there!

If for whatever reason you can’t make it, as usual feel free to reach out if you’re interested and would like to learn more.

@TomHCAnderson

 

About Tom H. C. Anderson Tom H. C. Anderson is the founder and managing partner of OdinText, a venture-backed firm based in Stamford, CT whose eponymous, patented SAS platform is used by Fortune 500 companies like Disney, Coca-Cola and Shell Oil to mine insights from complex, unstructured and mixed data. A recognized authority and pioneer in the field of text analytics with more than two decades of experience in market research, Anderson is the recipient of numerous awards for innovation from industry associations such as CASRO, ESOMAR and the ARF. He was named one of the “Four under 40” market research leaders by the American Marketing Association in 2010. He tweets under the handle @tomhcanderson.