To evaluate an AI voice agent, judge it on real phone calls, not a demo: every possible call outcome, tested end to end, with each result checked inside your own CRM and a way to stop the AI on any lead in one step. That is the bar SayVo sets for itself. Before one of our agents calls a real lead, all 9 call outcomes it can produce are tested on a real phone, from booked to wrong number, and every one is traced into the client's CRM. This page is that standard written down, built from 200,000+ real calls, so you can hold any vendor to it, including us.
- A demo shows the best call. Ask to see the busy, voicemail and “not interested” calls too.
- Judge the voice on a real phone. People expect a reply within a fraction of a second.
- Every call outcome should land in your CRM: stage, tag, note, recording, owner.
- You need a one-step stop button, proven before launch, and a staged rollout.
Why does a demo call tell you so little?
A demo is one call, chosen by the vendor, to a person who wants it to go well. Your leads are not that person. They are busy, half-listening, driving, or sure it's spam. The hard part of AI calling is everything that happens on those calls, and a demo never shows it.
You are probably the sixth AI company we have interviewed, and most of them sound robotic.
Owner of a real estate and mortgage business, on a SayVo discovery call
The wider data says the same thing. MIT's NANDA initiative found that for 95% of companies in its dataset, generative AI was falling short.[1] The same research found that tools bought from specialized vendors succeeded about 67% of the time, while internal builds succeeded about a third as often.[1]Buying beats building, but only if you can tell a tested product from a good demo. (If you're weighing that choice, see enterprise voice AI vs done-for-you.)
So test it like you would test a new hire: on the calls that go wrong.
Does it sound like a person on a real phone?
Start with timing, because people notice it before they notice words. A study of conversation in 10 languages found the most common gap before someone replies is 0 to 200 milliseconds, with a median of about a tenth of a second.[2]The same study found replies that don't answer the question are often delayed by up to a full second.[2]So a long pause doesn't just feel slow. It sounds like an evasive answer.
Test on a real phone, not a browser widget. We see this often: a test widget can sound worse than the phone line, with pauses that never happen on a real call, and it can also hide problems. Call it from your own cell, in your car, with the radio on. Then listen for these:
Those last three are rules we enforce in every script. Each SayVo prompt carries a banned list of customer-service phrases that make a caller sound like a bot, and every line the AI says in testing is scanned for them by script. The no-repeat rule exists because parroting the caller is one of the fastest tells. For how the words are trained, see what training on 200,000+ calls means.
What happens on the calls that go wrong?
Most calls are not bookings. They are voicemails, busy signals, wrong numbers and people who say no. Each one needs a different next step, and a mistake on any of them costs you the lead or annoys someone who asked to be left alone. That is why our go-live check covers every outcome, not just the happy path:
The appointment lands on the right person's calendar and the alerts reach the owner and the rep.
The lead is flagged for a human and the follow-up starts.
The AI stops calling and the CRM says why.
The lead is moved out of the calling list, with the reason in the notes.
A callback is timed, not a vague "we'll try later."
The lead enters the follow-up sequence and stays in it. Both kinds are tested, because call screeners are common now.
The system retries instead of marking the lead as worked.
The number is flagged so nobody keeps dialing it.
Voicemail and AI voicemail count as two outcomes, which makes 9 in all. We repeat test calls until each one has been seen end to end. It works the same way in any business: an estimate follow-up for a home services company and an intake call for a law firm have their own versions of “booked” and “not interested,” and each needs the same proof.
Before those real-phone tests, we stress-test the script itself. The agent calls a second agent that plays the lead, with full, partial and empty lead details, each with an objection. One build went through six prompt versions in a single evening this way. It tests the script, not the voice, so we always finish on a real phone.
How does it handle “Is this a robot?”
Someone will ask. How the AI answers decides whether the call continues. Our default is simple: when a caller asks if it's AI, the agent says yes the first time, never denies it, and moves straight to the next step. It also treats “Are you real?” as the same question. We learned that one the hard way, when an agent answered “yeah, I'm real” and admitted it was an AI a minute later.
As soon as they realize AI, they're hanging up.
Owner of an independent mortgage brokerage, describing another AI caller
Ask every vendor to show you this moment on a real call. Ask too what disclosure options you get, because some businesses want the AI to say it up front. Rules for AI calls are also stricter than many owners expect. The FCC ruled in February 2024 that AI-generated voices count as “artificial” under the Telephone Consumer Protection Act.[3]Consent and calling rules are your business's responsibility, not the vendor's; our guide to TCPA rules for AI calling explains what to check with your counsel.
Does it sound like your business, not a generic bot?
A good agent should pass a test your own best salesperson would set. It should know what you sell, which questions to ask before it books, and what it must never say. Generic voice tools often get the first part right and miss the other two. For more on that gap, see industry-trained vs generic voice AI.
Every business has its own words to avoid. We keep a ban list for each client in its script, on top of the house list. One owner banned a casual filler word because, in their market, it “sounds like you're lying.” Another banned slang that felt too familiar for their callers. These sound small. On a call, they are the difference between “that was your assistant” and “that was a robot.”
Does every result land in your CRM?
A call that isn't recorded in your CRM didn't happen, as far as your team is concerned. For each of the 9 outcomes, check that the right stage, tag, note, recording link and owner show up on the lead record your team already opens. Then check it in the CRM itself, not the vendor's dashboard. A system can report “saved” and change nothing.
Two more checks catch problems late. First, the trigger: a new lead should start a call within 60 seconds. Second, the live switch: test connections should move to production last, only after every outcome works. Our CRM integration guide covers each system in detail, and the full list of tools we connect is on our integrations page.
Can you stop it on one lead, right now?
Every business eventually needs the AI to stop calling one person immediately, without emailing the vendor. If there is no clear way, your team will invent one, and it may silently do nothing. We have seen it: a team kept using a stop command the system didn't recognize, and the AI kept calling a lead they believed was stopped.
So the stop button is a launch step for every SayVo client, not an extra. We build it into the CRM the team already uses, prove it on a test lead through the live system, confirm the lead actually left every workflow, and walk the team through it on the launch call.
How should the launch be staged?
Never switch on your whole list at once. The safest launches start with older leads, where a mistake costs least, and grow in steps. That also trains the agent on real conversations before your newest, most expensive leads reach it.
10 to 20 older leads. Read every transcript.
100 to 200 leads. Read the results with the owner and fix what showed up.
New leads, called within 60 seconds, around the clock, with the stop button in place.
Before the go-live, we also audit the live systems, not our notes. When one build was copied from an earlier one, it carried six small flaws, and all six had passed earlier reviews. Reading the live settings line by line is what caught them.
I don't want to just turn this on and say, hey, it sucks and cut ties.
Owner of an independent mortgage brokerage, on a SayVo discovery call
What should you watch in the first two weeks?
Testing before launch is half the job. The other half is watching the first real calls closely, because real leads say things no test thought of. Here is what we watch on every new agent, and what you should expect any vendor to watch with you:
Fixes after launch should be small and surgical: change the one line that caused the problem, re-test the path it affects with a real call, and leave the rest alone. Rewriting a working script to fix one moment is how new problems get in.
What should you ask to see, beyond the demo?
| Check | What a demo shows | What to ask for |
|---|---|---|
| Voice | One polished call in a browser | A live call to your own cell, where you interrupt and push back |
| Outcomes | A booking | Recordings of busy, voicemail, not interested and wrong-number calls |
| "Is this AI?" | Rarely comes up | The exact answer the agent gives, and your disclosure options |
| CRM | A dashboard | One test lead traced into your own CRM for each outcome |
| Stop button | Not shown | A stop you trigger yourself, proven before launch |
| Launch | "Turn it on" | A staged plan: small sample, a real batch, then full volume |
Questions to ask any AI voice agent vendor
For how a well-built agent runs a call from first ring to booking, see how the AI qualifies and books a lead. New to the category? Start with what an AI voice agent does. Comparing vendors side by side? Our platform comparison covers the field.
Where this breaks
Testing finds most problems, not all of them. Real leads say things no test script thought of, which is why the first batches get read line by line. Agent-to-agent tests check the script but not the voice. A browser test can make a good agent sound bad, and a scripted demo can make a weak one sound great. Even a stop button only works if the team knows it exists. None of this means AI calling is risky. It means the testing is the product, and you should know exactly how a vendor does it before you sign.
Get a live call from a SayVo agent on your own phone. Interrupt it, push back, ask if it's AI. Then book a demo and we'll walk you through how we'd test it on your leads.
Book a demoSayVo is the premium, done-for-you AI voice agent for independent mortgage brokerages. Built on 200,000+ real mortgage calls, it calls every new lead within 60 seconds, works inside your CRM and goes live in 2 to 4 weeks.
Frequently Asked Questions
You should. Ask for a live call to your own phone, recordings of the calls that don't book, and a test lead traced into your own CRM. At SayVo, every call outcome is tested end to end before an agent reaches a real lead.
Some will, as they do with human callers. What matters most is how the agent answers the question. Ours says yes the first time it's asked, never denies it, and moves straight to the next step, so the call keeps its purpose.
With a stop button inside the CRM your team already uses, such as a set task or tag. It should be proven on a test lead before launch. SayVo builds and proves one for every client and shows the team how to use it on the launch call.
Many do. Judge it on a real phone, not a browser widget: listen for fast replies, no repeating what you said, no help-desk phrases, and natural handling of names and numbers. A long pause before a reply is the most common giveaway.
At SayVo, testing is part of the 2 to 4 week build. The launch is then staged: a small sample of older leads, then a larger batch, then full volume once the results have been read with you.
Your business is. The FCC treats AI-generated voices as artificial under the TCPA, so consent rules apply. A good vendor will explain the rules and build to your choices, but confirm your approach with your own counsel.