Start an AI customer service pilot with one recurring question staff can answer from an approved source. Have the system draft the reply for staff to verify and send, and route uncertain or account-specific requests to the right person. Opening hours and required forms are practical starting points. Check where the source material falls short and whether staff can spot errors before a reply reaches a customer.
Pick a task with a clear source of truth
Take recent examples and sort them by which questions can be answered from approved guidance, which need an account check and which require a decision by a manager. At a hypothetical repair company, an assistant can draft the general rescheduling process using the published policy. A request to waive a fee should go to a colleague who can review the account and has authority to decide.
Limit what the system can see and do
- Use a small set of approved documents with a named owner and version date.
- Test with fictional or appropriately de-identified examples that omit information the answer does not need.
- Show the supporting passage so staff can verify a draft, edit it or discard it.
- Assign an escalation route for unclear questions, conflicting sources and policy exceptions.
Test mistakes and privacy boundaries before expansion
Test incomplete questions, outdated guidance, misleading case details and a request for someone else’s information. Check the reply, the sources the system accessed, what the logs retain and what a colleague receives at handover. NIST’s AI Risk Management Framework gives teams a structure for governing and assessing AI risks. OWASP identifies sensitive information disclosure as a risk in language-model applications.
Compare results with the existing workflow
Ask colleagues to handle the same sample with the current process and with assisted drafts. Record time spent reviewing, factual corrections, incorrect escalations and whether anyone accessed data they should not have seen. Select the next use case only when the team knows what the pilot improved and how it handles failures. If reviewing takes as long as writing the reply, improve the source material or narrow the topic and repeat the test.
Before the pilot, decide which events trigger a pause and name the colleague who can take that decision. A customer record appearing in the wrong case warrants stopping the pilot and reviewing the access rules. Make sure staff can continue handling enquiries through the existing process while the issue is investigated.
Frequently asked questions
Should the first trial reply directly to customers?
Start with staff reviewing and sending drafts. Use the pilot to establish which questions can be answered consistently from current, approved material.
Can we upload our full support history?
Begin by identifying what the pilot actually needs. Confirm permitted tools, access and retention with the people responsible before using customer records.
What should staff do if a draft has no evidence?
Have staff reject the draft and use the documented escalation route. Record the missing source so the knowledge owner can address it.

