What Do Your Customers Actually Think About Chatbots?

Chatbots today are what mobile apps were in 2012. Every business wants one. But what do consumers actually think of them?

That’s what we were wondering here at Userlike. We found studies on chatbot perceptions, but we weren’t convinced of their reliability. Most were published by companies with a clear interest in proving the benefit of chatbots, like chatbot building platforms.

cartoon of R2D2

At Userlike, we don’t have such a vested interest. We provide live chat software for human support teams, and we offer a simple chatbot and chatbot API for our customers to connect their sophisticated bot to.

Bots aren’t our main shebang, so in a sense we’re Switzerland; neutral, but curious about the fuss surrounding chatbots to see whether this is an area we should develop further in.

We decided to create a survey asking people to rate their past chatbot experiences and let us know what they liked and disliked. We’re not a research institute, so to make it easier for you to trust our results, we’re sharing the process behind our data and conclusions with as much transparency as possible.

Our methodology

We created our survey with Google Forms and paid 300 Euros to post it on Amazon’s Mechanical Turk, a crowdsourcing marketplace.

Respondents were paid to take our survey, so we created a handful of contradicting questions to filter out 'bad' surveys. If a respondent said they loved using chatbots in one question, and then said they hated them later on, we knew they were just in it for the moolah.

Question of whether you would chat with a chatbot for customer service
The chatbot love is strong with this one

We used SurveyMonkey’s sample size calculator to determine how many people we needed. Some questionnaires had to be rejected, but in the end we collected 100 quality responses before closing the survey.

Looking for better customer relationships?

Test Userlike for free and chat with your customers on your website, Facebook Messenger, and Telegram.

Read more

The goal was to learn if respondents had used a chatbot before, how they felt about the experience, and the pros and cons of chatbot use. We also dug deeper to find out how the consumer views a company that employs a chatbot, and in which situations a chatbot is helpful. Some answers were expected but others were just plain surprising.

Demographics

The survey was 22 questions long and devoid of any confusing technical jargon to make it appropriate for everyone. We learned that majority of our respondents are male and between the ages of 28-37 years old.

Pie chart of age ranges.
Pie chart of gender distibution.

Not many baby boomers (only 11.9%) responded to the survey, but this is not very surprising. People aged 55 and over are less likely to use live chat, which may mirror chatbot usage.

The main takeaways

Many consumer perceptions we received were understandable and reasonable. This is what stood out to us from our results:

1
Nearly every respondent has interacted with a chatbot before

This is likely due to selection bias and majority of our respondents being young, but it’s surprising nonetheless. But what was even more surprising to us, is how positive the respondents felt about chatbots.

Pie chart of chatbot experience distribution.
Bar chart of positive and negative experiences with chatbots.

According to Entrepreneur, “Chatbots are giving people the critical engagement factor of welcoming and belonging.” People have expressed that they often feel unappreciated by the companies they do business with, and a chatbot could bridge the communication gap.

2
People prefer instant chatbot support to waiting for a human agent

When given the choice, a consumer would rather speak to a chatbot directly to solve an issue rather than wait for an available agent.

Pie chart of preference to wait for human or instant chatbot support.

Voice of Customer (VoC) platform Usabilla’s research on customer interaction with AI supports this opinion. At least 54 percent of respondents said they would prefer to speak to a chatbot instead of a human customer service rep if it saved them 10 minutes.

3
It should always be easy to escalate to a human agent

Respondents may prefer to speak with a chatbot directly, but they still want an easy option to reach an agent just in case.

Additionally, our data showed that people would chat with a bot to connect with an agent who can assist with their specific inquiry. Perhaps it’s because respondents are familiar with phone trees, which ask a series of prompts before redirecting a call.

Bar chart about preference of being forwarded to a human.

4
People perceive chatbots as innovative rather than cheap

Chatbots have the potential to strongly reduce service costs for the company, so we were interested whether consumers would perceive companies that deployed them as cheap.

So we asked one question about the perception of cheapness, and one about innovativeness. And to our surprise, innovativeness was the clear winner. Chatbots might make things cheaper for the company, but that doesn't seem to reflect negatively on the company. Double win!

Bar chart showing that people don't see chatbots as cheap.
Bar chart showing that people regard chatbots as innovative.

5
Chatbots are fast and convenient, but have their limits

Respondents like that chatbots are immediately available and respond quickly, but most people said that chatbots have limited intelligence and aren’t suitable for filing complaints or solving issues.

Negative aspects of chatbots.

6
People trust chatbots with basic requests

Respondents are willing to chat with a bot for simple inquiries (product specifications, order status, shipping). However, this begs the question: How much do people trust chatbots?

At least 36% of respondents would use a chatbot to pay a bill and 41% would use it to check an order status. Respondents aren’t willing to settle disputes through bots, but what level of personal information are they willing to share? Just account access, or more out of pure convenience?

Scenarios that people would use a chatbot for.

In addition, do people want more websites to use chatbots? Perhaps if the AI was spectacular, but this is just speculation.

7
Chatbots should never pretend to be human

More than half of our respondents (59%) want chatbots to make it clear that they’re a bot, which is completely fair. Bots should avoid entering the uncanny valley; it’s deceptive and downright creepy.

8
Chatbots should be easy and fun to talk to

Despite respondents saying that they want a chatbot to state it’s a bot, they also wish for it to sound more human. Even though this is incredibly contradictory to the previous point, consumers want their bot conversations to be natural and pleasant.

Others also said that chatbots should occasionally make jokes or show emotion.

  • Q: What would make your experience chatting with a chatbot more positive?
  • A: “Perhaps if they reacted in a bit more human fashion. If their responses were more true to life or they were more responsive to language cues.”
  • A: “More intelligent/surprising responses that made me feel it could be a real person.”
  • A: “I would have a more positive experience chatting with a chatbot if they were to exhibit more human traits in their manner of talking, for example if they inject humor into their conversation or if they were to be more compassionate.”
  • A: “A joke here and there between the conversation would be nice and make the conversation light and more humanistic.”

Humor is a nonsensical algorithm that shouldn’t matter, but it does. The argument is that chatbots are engaging in conversations and should therefore have a personality and sense of humor.

But until someone can create AI that organically crafts original witty jokes, it’s tough to expect most companies to develop their own funny chatbot. We can’t all create a Poncho (RIP) no matter how hard we try.

How reliable is our data on consumer chatbot perceptions?

Our sample of consumers responded favorably toward chatbots, which may translate into the greater population. There is a chance answers were skewed because it was a paid survey and people responded in a way they thought we wanted to hear.

Mechanical Turkers might also be more tech-savvy than the average person. To gain a broader insight, we are considering another research approach by collecting responses on the street.

A larger sample of +/- 300 instead of 100 respondents would also allow us to reduce our error margin from 10% to 5%.

Overall, chatbots are making a good impression – better than we expected. Our research could be improved but we can be fairly sure that most millennial consumers perceive chatbots positively.