AI Customer Support Risk Controls

When this matters

• An AI assistant can draft replies but should not issue refunds or change account status.
• A customer asks for deletion, medical guidance, financial advice, or legal interpretation.
• A bot is connected to helpdesk macros, billing context, or CRM actions.

How to run it

1. List the customer outcomes that create money, privacy, legal, or trust exposure.
2. Map each outcome to policy sources and human escalation rules.
3. Generate red-team prompts that attempt to trigger the risky outcome.
4. Score the bot for hallucination rate, action overreach, and escalation misses.
5. Prioritize repairs by severity and likely customer impact.

Common risks

• A polite answer can still be risky if it promises an unsupported customer outcome.
• The highest-risk prompts often look like normal frustrated customer messages.
• Without monthly regression, risk can rise after innocent policy edits.