# Evidence Matrix for an AI Receptionist Acceptance Test

A useful acceptance test connects each caller job to evidence outside the
conversation. The matrix below keeps voice quality separate from the business
result. Use fictional caller data, a test number, and a calendar or inbox owned
by the test team.

| Caller job | Expected result | Evidence source | Failure example |
|---|---|---|---|
| Ask for business hours | Answer matches approved facts | Business profile and transcript | Invented holiday hours |
| Leave a callback request | One usable request reaches the owner | Team inbox or CRM task | Missing callback number |
| Book an appointment | Exactly one supported slot is created | Destination calendar | Verbal confirmation without an event |
| Reach a person | Call connects or enters the defined fallback | Call record and destination phone | Silence after transfer starts |
| Report an urgent issue | Correct urgency and owner are recorded | Queue item and notification | Every request marked urgent |

## Test the information boundary

Ask one question whose answer exists in the approved business facts and one
whose answer does not. The first should be accurate. The second should produce
an honest limitation, a follow-up question, or a defined handoff. Confidence is
not evidence when the source information is absent.

Then correct a detail during the call. Change the spelling of a name or the
callback number. The final request should contain the corrected value once, not
both versions.

## Test external actions separately

For a booking, state the service, date, time, and timezone. Inspect the calendar
after the call and confirm the owner, duration, and number of events. Repeat
with an unavailable slot. A request for a time is not a confirmed booking.

For a transfer, use a destination controlled by the test team. Cover answered,
busy, declined, and timeout behavior. The caller must hear the configured
fallback when nobody connects. Verify the actual destination call record, not
only the receptionist's announcement.

## Test retries and duplicate events

Send the same completion event twice or replay a provider callback in a safe
test environment. The workflow should create one message, one task, or one
appointment. If the provider times out after accepting an action, the retry
must reuse the same idempotency key.

Record human takeover as a separate case. Once a person owns the request,
automatic callbacks and messages for that request should pause without
disabling follow-up for every other caller.

## Define the pass decision

Score each row as passed, failed, or not observed. "The call sounded good" is
not a pass condition. A row passes only when the expected result exists in the
named evidence source and no duplicate side effect exists.

Teams can run this matrix against a configurable receptionist such as
[SkipCalls](https://skipcalls.com) or another provider. Product features still
need to be configured for the specific business, and every external action must
be verified in its destination system.

Start with a limited forwarding window. Expand only after the common caller
jobs leave correct, repeatable evidence.