TapCub is live — analytics, insights and live chat, free on one platform
Live Chat

How to Choose a Chatbot Provider: Weighted Scoring, Data Questions and Cost

Choose a chatbot provider with a weighted matrix instead of feature lists. Criteria, a scored example where the ranking flips between two buyers, data-handling questions from GDPR Article 28, an accuracy test and a cost break-even.

Lin Zhou Published Nov 21, 2026 12 min read Live Chat
Sample data: with weights that favour data handling and answer control, Z leads; for buyer A the order is Y, X, Z. Moving 10 weight points from data handling to cost cuts Z's lead over Y from 0.45 to 0.05.Sample data

To choose a chatbot provider, weight your needs so they add to 100, score each shortlisted provider 1 to 5 on the same criteria, compute Σ(weight × score) ÷ 100, and check whether the winner survives a 10-point shift in weights. In the example, the same three providers rank Y, X, Z for a small store and Z, Y, X for a regulated EU service; no provider wins for everyone, only for a set of weights.

What you will learn

  • Weights express your needs: the same scores gave 3.75 for provider Y with buyer A weights and 3.45 with buyer B weights, while Z moved from 2.85 to 3.90
  • If a 10-point weight shift changes the winner (a 0.45 lead became 0.05), treat the providers as tied and decide on the trial
  • Test accuracy on 30 real questions and watch wrong answers on the hard 10; a chatbot's answers are your statements

How do you choose a chatbot provider? Score them against your needs, not their feature lists

Choose a chatbot provider by writing down what you need, weighting each need, scoring each shortlisted provider 1 to 5 against the same needs, and checking whether the winner survives a change in the weights. The reason to do it as arithmetic is that vendor comparison pages score every product on the vendor's own criteria. A weighted matrix puts your priorities in the formula, and shows you when two providers are effectively tied.

This guide gives you the criteria, a scored example for two very different buyers where the ranking flips, the data-handling questions you can ask based on what GDPR Article 28 requires of a processor, a way to test answer accuracy before signing, a cost model with a break-even point, and a two-week pilot plan. No providers are named: the process is what transfers, and any specific product comparison would be out of date in a quarter. For a list of tool types, see website chat tools; for what chatbots cost in total, see how much do chatbots cost.

Which criteria should you compare?

CriterionWhat to askEvidence to request
Answer controlCan the bot answer only from content I supply, and can I see which source it used?A demo with your own documents; a list of questions it declined
Human handoffWhen it cannot answer, does a person get the full conversation?A test chat that escalates mid-conversation
Data handlingWhere is data stored, how long, who are the sub-processors, can I delete it?The data-processing agreement; sub-processor list; retention settings
ReportingCan I see resolved, handed-off and unanswered questions?A sample report on real data, not a marketing dashboard
CostWhat is the monthly price, and what changes it as volume grows?A written price at your volume and at three times your volume
Setup effortHow many hours to the first live answer, and who maintains it?A time estimate from someone who has done it, plus a named owner on your side
IntegrationsWhich systems does it read from or write to, and who builds that?The list, with what is included and what is custom
LanguagesWhich languages does it answer in, and are they tested?A test in each language you serve

Not every row matters to every buyer. The next section shows how weights express that.

ProviderBuyer A (small store)Buyer B (regulated EU service)
X: template bot3.452.75
Y: live chat plus AI3.753.45
Z: custom build2.853.90

Weights for A favour cost and setup; weights for B favour data handling and answer control. Totals = Σ(weight × score) ÷ 100.

Worked example: a weighted matrix for two buyers

Method: give each criterion a weight so the weights add up to 100, score each provider 1 to 5 on each criterion, then compute Σ(weight × score) ÷ 100. A total of 5.0 would mean a perfect score everywhere.

Sample data: three anonymous providers, scored on six criteria after a trial.

CriterionProvider X: template botProvider Y: live chat plus AIProvider Z: custom build
Cost542
Setup effort541
Human handoff243
Answer control235
Data handling335
Reporting243

Two buyers with different needs:

  • Buyer A: a small online store that wants something live this month on a low budget.
  • Buyer B: a regulated service in the EU that must control what the bot says and where data goes.
CriterionWeight, buyer AWeight, buyer B
Cost2510
Setup effort205
Human handoff2020
Answer control1525
Data handling1030
Reporting1010
Total100100

Computed totals:

ProviderBuyer ABuyer B
X: template bot3.452.75
Y: live chat plus AI3.753.45
Z: custom build2.853.90

How one total is computed: for buyer A, provider Y = (25×4 + 20×4 + 20×4 + 15×3 + 10×3 + 10×4) ÷ 100 = (100 + 80 + 80 + 45 + 30 + 40) ÷ 100 = 3.75. For buyer B, provider Z = (10×2 + 5×1 + 20×3 + 25×5 + 30×5 + 10×3) ÷ 100 = (20 + 5 + 60 + 125 + 150 + 30) ÷ 100 = 3.90.

The same three providers rank Y, X, Z for buyer A and Z, Y, X for buyer B. No provider is the best provider; one is best for a set of weights.

Sensitivity check. Move 10 points of buyer B's weight from data handling (30 to 20) to cost (10 to 20). Provider Z falls from 3.90 to 3.60 (it loses 10 × 5 on data and gains 10 × 2 on cost) and provider Y rises from 3.45 to 3.55 (loses 10 × 3, gains 10 × 4). The lead shrinks from 0.45 to 0.05. If a result flips after a 10-point shift, treat the two providers as tied and decide on the trial, not the matrix.

app.tapcub.com

Weighted score, buyer B

out of 5.0
Z: custom build3.90
Y: live chat plus AI3.45
X: template bot2.75
Sample data
Sample data: with weights that favour data handling and answer control, Z leads; for buyer A the order is Y, X, Z. Moving 10 weight points from data handling to cost cuts Z's lead over Y from 0.45 to 0.05.

What data-handling questions should you ask?

If the bot handles personal data about people in the EU, GDPR Article 28 sets out what must be in the contract between you (the controller) and the vendor (the processor). Turn that into questions:

  1. Instructions. Does the vendor process data only on your documented instructions?
  2. Sub-processors. Will they get your prior written authorisation, specific or general, before using another processor? With a general authorisation they must tell you about planned changes so you can object. Ask for the current list.
  3. Security and confidentiality. Are staff bound by confidentiality and are the Article 32 security measures in place?
  4. Data subject requests. Will they help you answer access and deletion requests?
  5. Deletion or return. At the end of the contract, will they delete or return all personal data, at your choice?
  6. Audits. Will they provide information to show compliance and allow audits, including inspections?
  7. Liability down the chain. The initial processor stays fully liable to you if a sub-processor fails.

A vendor that cannot answer these in writing has told you something. This is a checklist, not legal advice; ask your own counsel to review the agreement.

How do you test answer accuracy before you sign?

A chatbot's answers are your statements. In Moffatt v. Air Canada (2024 BCCRT 149), British Columbia's Civil Resolution Tribunal held an airline responsible for incorrect information its website chatbot gave a customer. Treat accuracy as a contractual risk, not only a quality metric.

Run this test with each shortlisted provider using your own content:

  1. Collect 30 real questions from your support inbox: 20 the content can answer, 5 it only partly answers, 5 it cannot answer at all.
  2. Mark each answer as correct, partly correct, wrong or declined.
  3. Calculate: correct answers ÷ 20 answerable questions; and wrong answers ÷ 10 unanswerable and partly answerable questions. The second number is the one that matters. A bot that confidently answers a question it should have declined is where liability comes from.
  4. Set a bar before you start: for example at least 17 of 20 correct (85%), and at most 1 wrong of the 10 harder questions. Adjust to your risk, but write it down first.

A provider with 18 of 20 correct and 4 of 10 wrong on the hard set is worse than one with 16 of 20 and 0 of 10 wrong, because the second declines when it does not know.

How does handoff to a human work, and how do you test it?

Ask for three things in the trial:

  • Triggers. What sends a chat to a person: a visitor asking for one, low-confidence answers, or a keyword? Can you change it?
  • Context. Does the agent see the whole conversation and the page the visitor was on, or does the visitor have to repeat themselves?
  • Hours. What happens outside working hours: a message form, or a bot that keeps promising a reply?

Run the test yourself: start a chat, ask something outside the knowledge, and time how long it takes to get to a person with the transcript in hand.

What is the cost trade-off, and where is the break-even?

Providers charge in different shapes: a flat monthly fee, a fee per resolved conversation, or a mix. Compare at your volume.

Sample data: Provider P charges $400 a month plus $0.50 per resolved conversation. Provider Q charges $0 a month and $1.50 per resolved conversation.

Resolved conversations per monthProvider PProvider Q
200$500$300
400$600$600
1,000$900$1,500

Break-even: 400 + 0.50x = 1.50x, so x = 400 resolved conversations a month. Below 400, Q is cheaper; above, P is. At 1,000 conversations P costs $0.90 each against $1.50 for Q. Add setup and upkeep hours (your time, valued at an hourly cost) to both before deciding.

What does a two-week pilot look like?

DaysTaskOutput
1–2Load the same 20 documents into each shortlisted providerSame inputs
3–4Run the 30-question testAccuracy and wrong-on-hard counts
5–6Test handoff and out-of-hours behaviourTime to human
7–8Ask the data-handling questions in writingWritten answers
9–10Score the matrix with weights agreed before the trialTotals
11–14Run the sensitivity check; run the winner live on a low-traffic pageDecision

How TapCub's chat fits this checklist

TapCub's live chat is one kind of candidate: a human live chat that adds AI replies from a knowledge base, so it should be judged against the criteria above like any other.

  1. Answer control. In Live chat › Knowledge base, question-and-answer entries are sent as written without calling the AI; document entries are searched and used to draft a reply. Use the Playground to test without spending quota.
  2. Handoff. In Live chat › Chat settings › AI assistant, choose off, AI only when all agents are offline, or AI first (it stops when an agent takes over, replies, or the visitor asks for a person). With too little material, it hands over to a person.
  3. Reporting. Live chat › Reports shows conversations, resolution rate, first response and knowledge-base hit rate (the share answered by direct match or retrieval).
  4. Boundaries. AI replies only generate text. They do not change orders, send emails or call other systems, and the chat widget has no pre-chat form, so ask for an email in the conversation or use offline messages.

If you need a bot that takes actions such as refunds or order changes, TapCub does not do that; it measures chat outcomes and conversion once you have one in place.

FAQ

What should I look for in a chatbot provider?

Answer control, human handoff, data handling, reporting, cost at your volume, setup effort, integrations and languages. Weight them by your needs and score each provider on the same scale.

How do I compare chatbot providers fairly?

Use a weighted matrix: weights add to 100, scores 1 to 5, total = Σ(weight × score) ÷ 100. Then shift 10 points between two weights; if the winner changes, the providers are effectively tied.

What questions should I ask about data?

Whether data is processed only on your instructions, who the sub-processors are and how you are notified of changes, how data is deleted or returned at contract end, and whether audits are allowed. These mirror GDPR Article 28.

Is a company responsible for what its chatbot says?

In Moffatt v. Air Canada (2024 BCCRT 149) a tribunal held the airline responsible for incorrect information from its website chatbot. Test accuracy and wrong-answer rates before you go live.

How long should a chatbot trial be?

Two weeks is enough to load content, run a 30-question accuracy test, test handoff and score the matrix. Longer trials mostly delay the decision.

Sources

Filed underLive Chat

Was this article helpful?

We use your answers to fix and expand this page.

Lin Zhou

Product Experience Specialist

Writes about product experience, practical workflows, and growth, turning complex processes into clear steps for everyday decisions.

See how a knowledge-base chat answers and hands off

Question-and-answer entries sent as written, document entries for drafted replies, three AI modes and a report with knowledge-base hit rate.

See your data clearly. Find your growth.

Every click, backed by data. Install one line of code and see your first numbers in a minute.

No credit card · Free plans for analytics and chat · Cookieless analytics