Evidence Matrix for an AI Receptionist Acceptance Test

publicv1
1h ago
1 views0 comments0 reviews3 min read
raw .md ↗

A useful acceptance test connects each caller job to evidence outside the conversation. The matrix below keeps voice quality separate from the business result. Use fictional caller data, a test number, and a calendar or inbox owned by the test team.

Caller jobExpected resultEvidence sourceFailure example
Ask for business hoursAnswer matches approved factsBusiness profile and transcriptInvented holiday hours
Leave a callback requestOne usable request reaches the ownerTeam inbox or CRM taskMissing callback number
Book an appointmentExactly one supported slot is createdDestination calendarVerbal confirmation without an event
Reach a personCall connects or enters the defined fallbackCall record and destination phoneSilence after transfer starts
Report an urgent issueCorrect urgency and owner are recordedQueue item and notificationEvery request marked urgent

Ask one question whose answer exists in the approved business facts and one whose answer does not. The first should be accurate. The second should produce an honest limitation, a follow-up question, or a defined handoff. Confidence is not evidence when the source information is absent.

Then correct a detail during the call. Change the spelling of a name or the callback number. The final request should contain the corrected value once, not both versions.

For a booking, state the service, date, time, and timezone. Inspect the calendar after the call and confirm the owner, duration, and number of events. Repeat with an unavailable slot. A request for a time is not a confirmed booking.

For a transfer, use a destination controlled by the test team. Cover answered, busy, declined, and timeout behavior. The caller must hear the configured fallback when nobody connects. Verify the actual destination call record, not only the receptionist's announcement.

Send the same completion event twice or replay a provider callback in a safe test environment. The workflow should create one message, one task, or one appointment. If the provider times out after accepting an action, the retry must reuse the same idempotency key.

Record human takeover as a separate case. Once a person owns the request, automatic callbacks and messages for that request should pause without disabling follow-up for every other caller.

Score each row as passed, failed, or not observed. "The call sounded good" is not a pass condition. A row passes only when the expected result exists in the named evidence source and no duplicate side effect exists.

Teams can run this matrix against a configurable receptionist such as SkipCalls or another provider. Product features still need to be configured for the specific business, and every external action must be verified in its destination system.

Start with a limited forwarding window. Expand only after the common caller jobs leave correct, repeatable evidence.

comments (0)

reviews (0)