AI Chatbot Launch Testing Checklist for Your Business

Set acceptance conditions for the actual business task

Write a test matrix around what the assistant must do for your visitors and team. A lead assistant may need to explain approved services, collect a valid contact method, record a confirmed enquiry and offer an appropriate handoff. An authenticated support assistant needs additional conditions for private data and customer-specific actions. For each case, record the input, the expected business outcome, the evidence and who approves the result. Decide which failures prevent launch and which can be addressed after a limited rollout. Google recommends defining acceptable evaluation criteria before testing an AI application; choose criteria relevant to your own workflow rather than borrowing an unrelated demo score.

Check the knowledge and the answers together

Prepare representative questions from your website, sales conversations and support records, using anonymised material where needed. Include simple questions, comparisons, misspellings, several questions in one message and follow-ups that refer to earlier answers. Check both the final response and the approved source it should use. Test a service with missing pricing and an outdated document alongside the current version. The assistant should not convert missing information into invented credentials, testimonials, delivery dates or terms. Record how content conflicts are resolved and who can publish a correction. A knowledge update should trigger review of the affected answers before the change reaches customers.

Exercise the complete lead capture journey

Test a visitor who supplies a name and email together, one who provides details gradually, and one who chooses phone instead. Add an invalid email, an incomplete phone number, a correction after submission and a visitor who declines to share details. Check that the application enforces the agreed minimum fields and does not invent missing information. Confirm the actual record in the test CRM or spreadsheet, including service interest, source and assignment. Reopen the chat or revisit the page to evaluate whether the session behaviour matches the documented design. A lead should count as captured according to the agreed backend result, rather than because the conversation merely mentioned contact details.

Test scope boundaries and access to private information

Try ordinary questions outside the assistant’s purpose and requests to reveal its configuration, another customer’s records or confidential business information. Include text that asks it to ignore its instructions and instructions embedded inside retrieved documents. The application should treat visitor and document text as input to evaluate, not as authority to grant access or change permissions. For assistants with tools, verify access rules on the server and validate action arguments independently of the generated response. Test signed-out access and the wrong customer account where applicable. A useful test outcome records what was blocked or handed over; it does not establish that the system can never encounter a new failure.

Break the model and integration dependencies deliberately

Use a controlled test environment to simulate a slow model response, a provider quota error, an unavailable CRM, malformed structured output and an interrupted network request. Verify that loading ends, the visitor receives understandable feedback and a retry does not silently create duplicate records. If you support a fallback provider, run the representative conversation cases against that provider too; the same prompt can produce different behaviour. Test timeout recovery and the distinction between an accepted request and a confirmed save. Your team should be able to identify a failed submission and follow the documented recovery process without copying customer secrets into a public support ticket.

Use the widget on real mobile and keyboard journeys

Open the assistant on a small screen with the on-screen keyboard visible. Check that the input, send control, close button and newest reply remain usable, and that the panel does not hide the only route to the underlying contact form. Navigate with a keyboard and review the focus order, accessible names and feedback for new messages. Respect reduced-motion preferences where the interface includes animation. Test a long answer, a long contact entry and an error message as well as the empty state. Measure the website before and after the widget loads so you can identify script, rendering or layout costs that need attention in the agreed release.

Review measurement without exporting customer details

Separate usage signals from business outcomes: widget opens, conversations started, leads submitted, destination saves confirmed and handoffs requested answer different questions. Agree their definitions before building a dashboard. Use identifiers and non-personal event categories where appropriate, keeping names, emails, phone numbers and conversation text out of general analytics payloads. Inspect test requests to verify the actual data sent. Confirm that failed submissions are not counted as successful leads and that retries follow the chosen counting rule. Access-controlled application logs may need additional operational detail, so define their purpose, permitted viewers and retention separately from visitor analytics.

Prepare an operating owner and a reversible launch

Name the people responsible for knowledge changes, provider accounts, failed records and customer escalation. Give them a runbook covering configuration, monitoring, the human contact route and how to pause the assistant while leaving the website usable. Start with an agreed limited scope, review real conversation patterns and recheck affected cases after changes to prompts, models or integrations. Google’s guidance treats evaluation and monitoring as continuing work because repeated tests can reveal different model outputs. Keep a record of release decisions and known limitations. EidenTech can help scope the chatbot, website experience and integrations around this checklist, with acceptance evidence tied to your own business process.

Relevant EidenTech services

Continue planning your project