Try it on your own data
Every report in this article is in the free plan. One snippet for web and apps, up to 10 sites.
Start freeA customer service chatbot should take the questions that are frequent, have a fixed answer and are cheap to get wrong, and hand everything else to people quickly. It answers, collects details and routes conversations; it does not judge exceptions. Below: how to pick the first questions, a recomputed volume example, launch steps, handoff rules and the metrics that separate resolution from deflection.
What you will learn
- Score questions by frequency, fixed answer and cost of error; in the sample three types made up 51% of volume
- Measure resolution, not deflection: 50% of conversations ended without handoff but only 39% were resolved
- Write the handoff rules before launch, and review a weekly sample of bot answers for errors
What does a customer service chatbot actually do?
A customer service chatbot answers customers' questions in a chat window without a person typing the reply, and passes the conversation to a person when it cannot. The software behind it can be a rule-based flow with buttons, a search over a set of prepared answers, or a language model that drafts replies from your documents. Which of these you pick is a separate question, covered in types of chatbots; the point here is what the bot should be given to do.
In practice a support bot does three jobs, and each has different limits:
- Answer. Give the same correct answer to a question that has one correct answer: delivery times, return windows, opening hours, how to reset a password.
- Collect. Gather what an agent will need (order number, email, a description) so the person who takes over does not start from zero.
- Route. Decide who should handle the conversation, and when it should leave the bot.
What it should not do is judge. Anything that depends on a person's circumstances, such as a refund exception, a complaint or a billing dispute, needs a human. The core of this guide: a customer service chatbot should take the questions that are frequent, have a fixed answer and are safe to get slightly wrong, and hand the rest to people quickly.
Which questions should the bot take first?
Score each type of question on three things: how often it comes up, whether the answer is the same every time, and what it costs when the bot is wrong. A question that is frequent, fixed and cheap to get wrong goes to the bot first. One that is rare, variable or expensive to get wrong goes to a person.
| Question type | Frequent? | Fixed answer? | Cost of a wrong answer | Who handles it |
|---|---|---|---|---|
| Shipping times and costs | Yes | Yes, by region | Low | Bot |
| Return and refund policy (what the policy says) | Yes | Yes | Medium, so quote the policy exactly | Bot |
| How to reset a password or log in | Yes | Yes | Low | Bot |
| Where is my order | Yes | No, needs a live lookup | Low | Bot gives the tracking link; person if stuck |
| Product compatibility or sizing | Medium | Partly | Medium | Bot with a person as back-up |
| Refund exceptions and billing disputes | Medium | No | High | Person |
| Complaints | Low | No | High | Person, quickly |
| Legal, medical, safety questions | Low | No | Very high | Person |
Two points that the table makes. First, a bot that can only produce text cannot look up an order or change an account, so "where is my order" is a link to a tracking page, not an answer. Second, "what the policy says" and "whether I get an exception" are different questions; the bot can do the first and should hand over the second.
| Question type | Fixed answer? | Cost of a wrong answer | Who handles it |
|---|---|---|---|
| Shipping times and costs | Yes | Low | Bot |
| Return policy (what it says) | Yes | Medium | Bot |
| Password or login help | Yes | Low | Bot |
| Where is my order | No, needs lookup | Low | Link, then a person |
| Refund exceptions, billing disputes | No | High | Person |
| Complaints, legal or safety questions | No | High to very high | Person, quickly |
A bot that only writes text cannot look up or change an order.
Worked example: how much volume can a bot take?
Sample data: a store samples 400 recent conversations and tags each by what the customer asked. All numbers are invented for the example.
| Question type | Conversations | Share of 400 | Fit for bot |
|---|---|---|---|
| Shipping times | 96 | 24% | Yes |
| Return policy | 72 | 18% | Yes |
| Order status | 64 | 16% | Link only |
| Product compatibility | 48 | 12% | Partly |
| Billing disputes | 40 | 10% | No |
| Password or login | 36 | 9% | Yes |
| Complaints | 24 | 6% | No |
| Other | 20 | 5% | No |
The three clear fits are shipping, returns and login: 96 + 72 + 36 = 204, which is 204 ÷ 400 = 51% of conversations. A bot will not resolve all of them; customers phrase things in ways the answers miss. Suppose it fully resolves 75% of the eligible questions:
- Share of all conversations resolved by the bot: 51% × 75% = 38.25%
- For 3,000 conversations a month: 3,000 × 38.25% = 1,147.5, about 1,148 conversations
- At 6 minutes of agent time per conversation: 1,148 × 6 ÷ 60 = about 115 hours a month
Now check that against a real month, which is where most business cases go wrong. Suppose, after launch, 1,500 of 3,000 conversations ended without a handoff, and 330 of those customers came back about the same problem within seven days:
- Conversations that ended without a handoff: 1,500 ÷ 3,000 = 50%
- Resolved conversations: 1,500 − 330 = 1,170, so 1,170 ÷ 3,000 = 39%
The plan said 38%, and the month shows 39%, so the scope was about right. The 50% figure would have overstated success by eleven points, because customers who gave up or came back are counted as handled. Put the conversion into money with the method in chatbot ROI; the broader picture of which support work can be automated is in customer service automation.
3,000 conversations → resolved
sample monthConversations
3,000
Ended without handoff
1,500
Resolved (no repeat contact)
1,170
How do you launch a support chatbot, step by step?
- Pull 200 to 400 real conversations. Use the last one or two months, not a list of questions you imagine customers ask. Real conversations show the real wording.
- Tag them. Give each a question type, as in the example, and count. Stop at 8 to 12 types.
- Write the answers for the top ten types. One question, one answer, in the words your best agent would use. Include the policy exactly as it is written, with a date; a bot that paraphrases a policy wrongly creates a promise you did not make.
- Write the handoff rules before launch (next section). The bot's edges are where customers are most likely to be frustrated.
- Test with questions it has not seen. Take 50 recent conversations that were not used for writing answers and replay the customers' first messages. Count how many it answers correctly, how many it hands off and how many it gets wrong. Fix the wrong ones before going live.
- Soft launch. Start with one page, one language or only outside working hours, and read every conversation for the first week.
- Review weekly. Look at handoffs by reason, wrong answers and the questions that arrive with no matching answer. Turn the good agent replies into new answers.
When should the bot hand off to a person?
A customer should always be able to reach a person, and the bot should offer it before the customer asks for the third time. The US Consumer Financial Protection Bureau describes the failure as a "doom loop" in which customers are stuck with a bot and cannot reach a live representative; it also notes that chatbots can give wrong answers. A hand-off rule is the remedy.
| Trigger | What the bot does | Why |
|---|---|---|
| The customer asks for a person | Hand off immediately, no further questions | Respect the request |
| Two answers in a row did not help | Offer a person | Avoid the loop |
| No answer matches | Say so; hand off or take a message | A guess is worse than a delay |
| Sensitive topic (complaint, billing dispute, safety) | Hand off with the topic noted | Cost of a mistake is high |
| Out of turns | Hand off | Conversations should not run indefinitely |
| Nobody is online | Take a message with an email address | Do not leave the customer waiting |
A good handoff also carries the context: what was asked, what the bot answered and what the customer has told it. If the customer has to repeat everything, the handoff has failed even if it happened on time. Liability matters as well: in Moffatt v. Air Canada (2024), a Canadian tribunal held the airline responsible for wrong information its website chatbot gave a customer. Treat what the bot says as something your company has said.
Which metrics show a support chatbot is working?
Use resolution, not deflection. Deflection counts every conversation that ended without a handoff, including customers who gave up.
| Metric | Formula | What it shows |
|---|---|---|
| Resolution rate | (Conversations without handoff − repeat contacts) ÷ all conversations | Real self-service |
| Handoff rate, by reason | Handoffs of each reason ÷ all conversations | Where the bot's edges are |
| Repeat contact rate | Customers back about the same issue within 7 days ÷ conversations without handoff | Hidden failure |
| Answer hit rate | Questions answered from the knowledge base ÷ all questions | How well content covers demand |
| Wrong-answer rate | Wrong answers in a weekly sample ÷ sampled bot answers | Safety of the content |
| Satisfaction | Rating of bot-handled conversations vs. person-handled | Whether customers accept the bot |
Check the wrong-answer rate by reading a sample, say 30 bot-handled conversations every week; no dashboard replaces that. Compare the satisfaction of bot-handled and person-handled conversations on similar question types, otherwise easy questions flatter the bot.
Which mistakes make support chatbots fail?
- Launching on a guess of what customers ask. The example shows the top three types make up 51% of volume; you only know that from real conversations.
- Letting the bot answer everything. Complaints and billing disputes are 16% of the sample, and each one needs a person.
- Measuring deflection. In the example, 50% looked like success and 39% was the truth.
- No path out. A bot without a visible way to reach a person traps customers.
- Stale answers. A policy changed last month and the bot still quotes the old one; put an owner and a review date on each answer.
- No review of wrong answers. Without a weekly sample, errors stay hidden until a customer complains.
How do you set this up in TapCub live chat?
TapCub's live chat has the three parts of the example: a knowledge base, an AI assistant that answers from it, and a handoff to agents. The assistant only produces replies; it does not look up or change orders, send email or call other systems, which matches the "link, not answer" treatment for order status above.
- Write the answers. In Live chat › Knowledge base, add Q&A entries for your top question types. When a visitor's message matches a Q&A entry, the stored answer is sent as written and does not use an AI call. Add documents (such as a policy page) for longer material; they are split into passages and the model drafts a reply from them. Draft inbox can generate draft Q&A entries from conversations resolved in the last 30 days; nothing goes live until you approve it.
- Pin the top questions. Pinned Q&A entries appear in the visitor-facing FAQ. Opening an answer does not create a conversation, and only Not solved or a request for a person brings in an agent.
- Choose the mode. In Live chat › Chat settings › AI assistant, pick off, only when all agents are offline, or AI answers first. The visitor is told they are talking to an AI, and replies are labelled. Set Max AI turns per conversation (1 to 20, default 6) and write the Handoff message.
- Review handoffs and fix answers. In Live chat › Inbox, each handoff is labelled with its reason: no knowledge match, the model could not answer, the visitor asked, the turn limit, or an agent took over. Use Fix & save on a wrong answer to store the corrected reply as a Q&A entry.
- Read the numbers. Live chat › Reports shows conversations, resolution rate (resolved ÷ all conversations), first response time, satisfaction and knowledge base hit rate. TapCub does not count repeat contacts for you; tally them from conversations tagged as returning, as the example did with 330.
FAQ
What is a customer service chatbot?
It is software that answers customers' questions in a chat window and passes the conversation to a person when it cannot. It can be a button-based flow, a search over prepared answers, or a language model drafting replies from your content. What it should handle is the frequent, fixed-answer questions.
What can a customer service chatbot do, and what can't it do?
It can give consistent answers to policy and how-to questions, collect details for an agent, and route conversations. It cannot judge exceptions, handle emotional complaints well, or act on an account unless it is connected to your systems. If a wrong answer is costly, a person should answer.
How do you set up a chatbot for customer support?
Pull 200 to 400 real conversations, tag them by question type, write answers for the top ten types, define when the bot hands off, test with unseen questions, soft launch on one page, then review handoffs and wrong answers weekly. The steps are in the launch section above.
How do you measure chatbot success in customer service?
Track the resolution rate, not deflection: conversations that ended without a handoff, minus customers who came back within 7 days, divided by all conversations. Add handoff reasons, repeat contacts, a weekly sample of wrong answers, and satisfaction compared with person-handled conversations.
When should a chatbot hand over to a human?
When the customer asks, after two unhelpful answers in a row, when no answer matches, for complaints and billing disputes, and when the turn limit is reached. Pass the context with the handoff, and make a person reachable even when nobody is online, for example through a message form.
See also: AI Live Chat: Set the Boundary Between Auto-Answers and Humans First.
See also: How Much Do Chatbots Cost? Pricing Models and Cost per Resolution.
Sources
- Consumer Financial Protection Bureau. Chatbots in consumer finance (2023) — limits of chatbots, wrong answers and "doom loops" with no route to a person. Verified 2026-11-19.
- Civil Resolution Tribunal of British Columbia. Moffatt v. Air Canada, 2024 BCCRT 149 — responsibility for information given by a website chatbot. Verified 2026-11-15.
Was this article helpful?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.

Product Analyst
Focuses on event analysis, conversion funnels, and retention, using clear metric definitions and structured analysis to explain product performance.
