---
title: "Test a voice agent with scripted scenarios"
description: "Build a test suite of scripted scenarios for a voice agent, run it, read the score, run history and trend, and apply a recommendation back to the agent's own settings."
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.speakai.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Test a voice agent with scripted scenarios

Every change you make to an agent, a new instruction, a rewritten personality, a knowledge base
update, can shift how it responds. The **Testing** tab is how you catch that before a real caller
does: you script the conversations you care about, run them against the agent, and get a score
back instead of guessing.


> **Run results aren't live yet**
>
> Selecting **Run tests** queues the run, but the engine that drives the simulated conversation
> against the agent isn't wired up yet, so a run stays **Queued** rather than reaching
> **Completed**, and won't produce a score or recommendations.

## What is a test suite?

A test suite is the set of test **scenarios** you've built for that agent. Open the **Testing**
tab on an agent that doesn't have one yet and you'll see **No test suite yet** with a **Generate
test suite** button. Select it and Speak AI reads the agent's own instructions and knowledge base
and writes you a starting set of scenarios with an LLM call, so you aren't building one from
scratch.

## Scenarios and criteria

Each scenario is one scripted mini-conversation, made of:

- A **Name** and a **Category**: Greeting, KB retrieval, Off topic, Edge case, or Custom, so you
  can see at a glance what kind of situation it covers.
- **User messages**, one per line, sent to the agent in order.
- **Criteria**, the checks the run scores that scenario against.

A criterion is one of four types. **LLM judged** asks an LLM judge to evaluate an evaluation
prompt you write, such as "Did the agent answer the user's question accurately?" **Response
length** fails if any of the agent's replies go over a maximum word count you set. **Regex match**
fails if none of the replies match your pattern, or, with **Must match** turned off, if any of them
do. **Tool called** fails if the agent never calls the tool name you specify. Every criterion also
has a **Weight** from 1 to 10, for how much it counts toward the scenario's score, and a
**Critical** flag you can turn on for the ones the agent has to get right.

You can add scenarios and criteria by hand with **+ Add scenario** and **Add criterion**, edit the
generated ones, or select **Regenerate suite** to have Speak AI write a fresh set. Regenerating
replaces the whole scenario list, so anything you've hand-edited goes with it. Select **Save
configuration** to keep your changes, along with a **Budget per run ($)** cap and two switches,
**Auto-run on KB update** and **Auto-run on instruction save**, so the suite can re-run itself the
moment you touch the knowledge base or the agent's instructions instead of waiting for you to
remember to check.

## Run a test

Once the suite has at least one enabled scenario, select **Run tests**. Only one run can be active
on an agent at a time, so wait for an existing run to finish, or cancel it, before starting another.

While a run is in progress, a status bar shows Queued, then Running scenario by scenario, then
Completed, along with how much of your budget it has spent so far and how long it has been
running. If you're an Owner or Admin, you also get **Pause**, **Resume**, and **Cancel** controls.
A cancelled run can't be resumed. A run can also end as Failed, if a step couldn't finish, or
Budget exceeded, if it would have spent more than the cap you set.

If the agent runs on a Live (speech to speech) model, a notice on this tab tells you tests run on
that model's text side and live audio isn't simulated.

## Read the results

Once a run completes, **Test results** shows its overall score as a percentage, how many scenarios
passed and failed, what the run cost, and what triggered it, manual, a knowledge base update, an
instruction save, or a schedule. If the score fell below the agent's best previous run, a
regression notice tells you and points at the scenarios that failed.

Select a scenario to expand it and see each of its criteria on its own, marked passed or failed,
with the judge's reasoning underneath where there is one. That's where you actually debug a
failing scenario: the top-level score tells you something regressed, the expanded criteria tell
you what.

## Run history and score trend

**Run history**, below the current results, lists every past run for the agent with its date,
trigger, score, change from your baseline, cost, and status. Select a row to open that run's own
results in the same view above, with a banner reminding you it's an older run and a link back to
the latest one.

**Score trend** charts your completed runs' scores over time, oldest to newest, with your
**baseline**, the agent's best completed run so far, drawn in as a reference line. History and
trend together are how you tell a one-off bad run from a real regression: a single dip that
recovers next run is noise, a trend that keeps sliding after an instruction change is worth a
closer look.

## Apply a recommendation

A completed run can come back with **Recommendations**: suggestions grouped by severity, high,
medium, or low, and by what they touch, Instructions, Personality, Knowledge base, Topics to
avoid, Custom vocabulary, Welcome message, or Response length. Filter by severity to focus on the
high ones first.

For a recommendation about the agent's instructions, personality, or topics to avoid, select
**Apply fix** and it's appended to that setting right away, no need to go find it in another tab.
For one about the knowledge base or vocabulary, you'll see **Copy fix** and a note to apply it in
the Knowledge / Vocabulary tab instead, so copy the suggested text and paste it in there yourself.
Either way, an applied recommendation gets an **Applied** badge with the date, and you can
**Dismiss** any recommendation you don't want to act on.

## Who can do this

Anyone with access to the agent can open the Testing tab and read its results, history, and trend.
Generating or editing the suite, starting, pausing, resuming, or cancelling a run, and applying a
recommendation all need the Owner or Admin role on your Speak AI account.

## Related pages

- [Run scripted tests against a Speak AI voice agent](/api/voice-agents/testing/): the API
  reference for the same suites, runs, and recommendations, useful if you want to trigger a run
  from your own backend or CI.
- [Voice and avatar](/help/voice-agents/voice-and-avatar/): how an agent's model, including a Live
  model, is chosen on the Details tab and how that changes what this tab can test.

Source: https://docs.speakai.co/help/voice-agents/testing/index.mdx