Choosing an AI chatbot for your business website: what to check first

There’s no single best AI chatbot, because the right one depends on what your customers ask and what would go wrong if the answer were wrong. For most small and mid-sized businesses, a good choice answers only from information you’ve approved, says so when it doesn’t know, hands the conversation to a person cleanly, and costs less to run than the time it saves. This guide covers how to tell which tools meet that bar.

The three kinds of website chatbot

Most of what’s sold as a “chatbot” falls into one of three groups.

A lot of businesses don’t need the third option. If a hosted tool can answer your common questions from your existing pages and pass the rest to your inbox, start there.

Why accuracy matters more than personality

AI chatbots write fluent answers whether or not those answers are right. Google puts it plainly in its guidance on generative AI content: “generative models don’t retrieve facts, but predict a likely sequence of words based on their training data.” A chatbot that sounds confident about your refund policy may simply be guessing.

That’s a business risk as well as a customer service one. In Moffatt v. Air Canada (2024), a BC Civil Resolution Tribunal decision, the airline’s website chatbot told a customer he could apply for a bereavement fare discount retroactively, which contradicted Air Canada’s actual policy. The airline argued the chatbot was responsible for its own actions. The tribunal called that “a remarkable submission” and found that “it should be obvious to Air Canada that it is responsible for all the information on its website,” as quoted in this summary of the decision. Air Canada was ordered to pay $650.88 in damages, plus interest and fees. The amount was small. The point for any business is that a chatbot on your site speaks for you.

What to check before you choose

Where its answers come from

Ask each vendor, or whoever is building it for you:

• Can you limit it to your own approved content (your service pages, policies and FAQs), or does it also answer from the model’s general knowledge?
• What does it do when it can’t find an answer? The right behaviour is to say so and offer a person. Making something up is the wrong one.
• When you update a page or a policy, how quickly does the chatbot reflect the change?
• Can you see every conversation afterwards, so you can catch wrong answers?

If your website doesn’t clearly state your hours, service area or policies, fix that first. An AI assistant will answer from whatever you give it, gaps included.

How it hands over to a person

Handover is where many chatbots frustrate people. Check what happens out of hours, whether the visitor can leave an email or phone number, and whether your team receives the full transcript so the customer doesn’t have to repeat themselves. If enquiries already land in a CRM, the handover should create a record there too. That part is ordinary workflow automation and often matters more than the AI itself.

What it must never answer

Write a short list of topics the chatbot should refuse and pass to a person. For most businesses that includes:

• prices, discounts or availability you haven’t published
• exceptions to your policies, refunds and complaints
• anything that amounts to legal, medical, financial or safety advice
• promises about results or timelines

Then test that it actually refuses them (more on testing below).

Privacy and transparency

Canada’s federal, provincial and territorial privacy regulators published joint principles for generative AI in December 2023. They interpret existing privacy law rather than adding new rules, and several apply directly to a website chatbot. For public-facing tools, the regulators say to “ensure that individuals interacting with the tool are aware that they are interacting with a generative AI tool.” They also say that, unless otherwise required, prompts “should not be retained, used for secondary purposes, or disclosed.”

In practice, that means you should know:

• where conversations are stored, for how long, and who at the vendor can see them
• whether the vendor or the model provider uses your conversations to train its models, and whether you can switch that off
• what your privacy policy needs to say about the chatbot
• whether visitors are told, up front, that they’re talking to an AI and should avoid sharing personal details they don’t need to

If your chatbot will handle health, financial or other sensitive information, get privacy advice before you launch.

Security, especially if it can take actions

A chatbot that only answers questions is lower risk than one that can book appointments, look up orders or issue refunds. The OWASP Top 10 for LLM Applications lists prompt injection as its first risk: a visitor, or text hidden in a page or file the chatbot reads, can push the model into doing something it shouldn’t. OWASP notes that no method is confirmed to fully prevent it. Its recommendations include restricting the model’s access “to the minimum necessary for its intended operations” and keeping a person in the loop for privileged actions.

For a small business, the practical rule is simple. Give the chatbot read-only access to what it needs, and have a person approve anything that changes a booking, an account or money.

Accessibility

In Ontario, private businesses and non-profits with 50 or more employees must make their public websites meet WCAG 2.0 Level AA, according to the province’s AODA guidance. A chat widget sits on those pages, so it’s sensible to hold it to the same standard: it should work with a keyboard alone, be readable by screen readers, and not cover content people need on a phone. Ask the vendor for its accessibility documentation, and try the widget yourself without a mouse.

What it costs to run

Pricing models differ, so compare them against your expected volume rather than the headline price:

• Per outcome or per resolution. Some platforms charge when the AI resolves a conversation. Intercom, for example, lists its Fin AI agent from $0.99 per outcome (the page doesn’t state a currency) and defines an outcome as, among other things, a customer confirming the issue is resolved or not asking for more help.
• Monthly plans with conversation limits. Predictable until you exceed the cap.
• Usage-based fees for a custom build. You pay the model provider per use, plus hosting and someone to monitor and maintain it.

Whatever the model, add the time your team will spend reviewing transcripts and updating content. As our AI integration page puts it, those ongoing costs should be estimated during scoping and weighed against the time the feature saves.

Test it before it goes live

A short, structured test tells you more than any demo. One way to do it:

1. Collect 20 to 30 real questions from your inbox, phone notes and reviews, in your customers’ own wording.
2. Add questions it should refuse from your “never answer” list, plus a few attempts to make it ignore its instructions.
3. Run them all and mark each answer as correct, wrong, or correctly handed over.
4. Fix the content or the settings behind each wrong answer, then run the set again.
5. After launch, read a sample of conversations every week for the first month.

If it can’t pass that test, it isn’t ready, however good the vendor’s demo looked.

When a chatbot isn’t the answer

If you only get a few questions a week, or most enquiries need a quote or a conversation anyway, a chatbot may add a step without adding much value. A clearer FAQ section, a better enquiry form and a quick, reliable reply process can serve customers better. A chatbot earns its place when the same questions come up often, the answers are already written down, and getting one wrong is low-risk or caught by a person.

When we scope an AI feature, we start with one specific task, define what a good answer looks like, and decide where a person reviews or takes over before anything is built. If you’re weighing a chatbot for your site, our AI integration service starts the same way.