Evidence Matrix for an AI Receptionist Acceptance Test
A useful acceptance test connects each caller job to evidence outside the conversation. The matrix below keeps voice quality separate from the business result. Use fictional caller data, a test number, and a calendar or inbox owned by the test team.
| Caller job | Expected result | Evidence source | Failure example |
|---|---|---|---|
| Ask for business hours | Answer matches approved facts | Business profile and transcript | Invented holiday hours |
| Leave a callback request | One usable request reaches the owner | Team inbox or CRM task | Missing callback number |
| Book an appointment | Exactly one supported slot is created | Destination calendar | Verbal confirmation without an event |
| Reach a person | Call connects or enters the defined fallback | Call record and destination phone | Silence after transfer starts |
| Report an urgent issue | Correct urgency and owner are recorded | Queue item and notification | Every request marked urgent |
Ask one question whose answer exists in the approved business facts and one whose answer does not. The first should be accurate. The second should produce an honest limitation, a follow-up question, or a defined handoff. Confidence is not evidence when the source information is absent.
Then correct a detail during the call. Change the spelling of a name or the callback number. The final request should contain the corrected value once, not both versions.
For a booking, state the service, date, time, and timezone. Inspect the calendar after the call and confirm the owner, duration, and number of events. Repeat with an unavailable slot. A request for a time is not a confirmed booking.
For a transfer, use a destination controlled by the test team. Cover answered, busy, declined, and timeout behavior. The caller must hear the configured fallback when nobody connects. Verify the actual destination call record, not only the receptionist's announcement.
Send the same completion event twice or replay a provider callback in a safe test environment. The workflow should create one message, one task, or one appointment. If the provider times out after accepting an action, the retry must reuse the same idempotency key.
Record human takeover as a separate case. Once a person owns the request, automatic callbacks and messages for that request should pause without disabling follow-up for every other caller.
Score each row as passed, failed, or not observed. "The call sounded good" is not a pass condition. A row passes only when the expected result exists in the named evidence source and no duplicate side effect exists.
Teams can run this matrix against a configurable receptionist such as SkipCalls or another provider. Product features still need to be configured for the specific business, and every external action must be verified in its destination system.
Start with a limited forwarding window. Expand only after the common caller jobs leave correct, repeatable evidence.