AI agent evaluation proposal with a test set from your own logs, release bars, a failure report, a retest and three one-time options.
Start free. No card required.
A starting point for the scope, pricing and next steps you want to share
Keep useful pages and replace the content with your client details
Choose the colors, fonts and imagery that fit your business
An AI agent evaluation proposal offers an independent test of an agent that already talks to customers: where it goes wrong, a test set with a release bar for each category, and fix support with a retest. This template is written for a travel company whose booking agent made costly mistakes, and offers three one-time options.
AI engineering and testing consultancies working with companies whose agent takes real actions for customers, such as bookings, refunds or orders. It suits an assessment before the agent is given more work, or after an incident that leadership wants explained.
Start from the client’s own conversation logs: how real conversations ended and where the agent got it wrong. Then show the test set by category, the release bar each category must meet, and how each case is run and explained, so engineers and leadership read the same results.
Price it as a one-time assessment, with options that add a retest after fixes and a test set wired into the client’s release pipeline. Split payment between acceptance and the leadership readout, and say that fixing the agent is the client’s work unless you include fix support.
Time to customise: about 20–30 minutes.
10 parts, in the order your client reads them. Every part is editable: keep what fits the job, rewrite the rest and add your own.
Your booking agent handles about 14,000 chat conversations a month and can change flights and hotel dates on its own.
From your conversation logs, the two incident reports and a session with your engineers.
Each case runs five times, because the agent does not always answer the same way. You see which failures block release and which can wait.
Everything in the recommended option, Assessment and retest.
Covers: Not included
We read your logs and incidents and write the 400 cases with your customer service lead.
Spent ten years in software testing before specialising in AI agents, and has assessed 15 agents in travel, insurance and banking since 2024.
An insurer’s claims agent, before it went to all customers.
Covers: The client, The result
Three options. We recommend Assessment and retest.
No test can cover every conversation. It shows how often the agent fails on the cases that matter most to you, and whether the fixes worked.
Pick an option and any extras, then accept on this page.
A structured test of an AI agent against real scenarios: a test set built from actual conversations, a pass mark for each category of task, and a report on where the agent fails and why.
Enough to cover each type of task the agent handles, including the rare, expensive ones. Several hundred cases drawn from real conversations is a common starting point.
The pass mark a category of tasks must reach before the agent is allowed to handle it, agreed in advance. A category below its bar stays with people until a retest passes.
Yes. Fixes can break other behaviour, so rerun the full test set after each change and compare the results category by category.
10 editable parts, in this order: Introduction, Your situation, Our recommendation, What is included, Timeline, Your team, Results for a similar client, Options and fees, Questions and Next steps. Keep the parts that fit the job, rewrite the rest and add your own.
As three one-time options the client can choose between: Assessment ($14,000), Assessment and retest ($21,000, recommended) and Assessment, retest and test pipeline ($30,000). Payment terms: 50% on acceptance, 50% at the readout. The prices are examples; replace them with your own before you send it.
Plan on about 20–30 minutes to replace the sample client, scope, dates and prices with your own, then preview it as your client will see it.
Share it as one link your client can read on any device, comment on, accept and e-sign, or export a PDF for your records.
Add your scope, pricing and brand. Preview the client experience before sharing.