Glowing AI agent dashboard with KPI gauge, funnel and trend chart, for the How to Measure AI Agent Performance guide

How to Measure AI Agent Performance: KPIs That Actually Matter

November 16, 2026•9 min read

An agency sends a client the first monthly report for their new AI agent. The headline says the agent handled 2,140 conversations. The client replies with one question: "Great, but did it get us any customers?" Nobody on the team can answer it.

This happens because most dashboards track activity instead of results. Conversation counts and message totals look impressive, yet they say nothing about whether customers got help or whether leads turned into sales. Some popular metrics even count frustrated customers as wins.

This guide explains which KPIs matter for support, lead and voice agents, how to calculate each one, what good results look like and how to put them into a simple monthly report.

Why Chat Volume and Containment Mislead

Containment rate is the share of conversations that never reach a person. Many tools show it first because it is easy to measure and usually high. The problem is that a customer who gives up and closes the chat also counts as "contained." According to research cited by Ada, up to 20% of contained tickets in one study were drop-offs where the bot had not helped.

A second issue is self-grading. Some platforms let their own AI decide which conversations count as resolved, and that label may also drive the monthly bill. That does not make the number wrong, but it does mean it needs an independent check.

The fix is simple. Pair every main KPI with a guard metric that would expose it if it were being gamed.

Start With the Agent's Job

The right KPIs depend on what the agent was built to do. A support bot, a lead capture bot and a phone agent succeed in different ways.

Custom HTML/CSS/JavaScript

Pick the row that matches your agent and treat everything else as supporting detail. An agent that does more than one job gets one main KPI per job.

Support Agent KPIs

Support is where most businesses start, and where AI customer support software usually takes the biggest load off a team. These five measures show whether that load is being handled well.

Automated Resolution Rate

Formula: conversations resolved without a person ÷ total conversations

This is the clearest sign that the agent is doing its job. Benchmarks vary widely depending on who publishes them. Intercom's Fin team calls 55% to 70% typical for routine support, while other vendors call anything from 30% to 60% good. Most of the gap comes from how each vendor defines a resolution.

Reopen and Repeat-Contact Rate

Formula: "resolved" conversations where the customer returns about the same issue within 48 hours ÷ all resolved conversations

This guard metric catches answers that sounded complete but did not solve anything. Intercom's Fin team treats a rate below 10% as strong. If resolution rises while reopens rise with it, the agent is closing conversations too early.

Handoff Rate and Handoff Quality

The handoff rate is the share of conversations passed to a person. A falling rate is good news only if the right chats still reach staff. Billing disputes, cancellations and anything involving safety should still reach a person quickly.

Quality matters as much as the number. Check a few handoffs each week and ask one question: did the team member get the full conversation, or did the customer have to start again? Passing full context at handoff is one of the oldest chatbot best practices, and still one of the most often skipped.

Unanswered-Question Rate

Formula: replies where the agent says it does not know ÷ total replies

This number points straight at gaps in the content the agent learns from. Group the unanswered questions by topic each month and add the missing answers through your AI knowledge base software. Review the list again the following month to confirm those topics have dropped off.

Sentiment and CSAT

Post-chat surveys reach only a small share of customers, and the ones who answer tend to be very happy or very upset. Sentiment tracking reads every conversation and tags it as positive, neutral or negative. Use both: CSAT for the customer's own words, sentiment for the full picture.

Lead and Booking KPIs

Sales-focused agents are judged by how many conversations move down the funnel. Each step below is measured against the one before it:

  1. Conversations to qualified leads: qualified leads ÷ conversations. Shows whether the agent asks the right questions.

  2. Qualified leads to bookings: bookings ÷ qualified leads. Shows whether the offer and booking flow work.

  3. Bookings to shows: attended appointments ÷ bookings. This is the guard metric, since an agent can book plenty of calls that never happen.

  4. Shows to customers: new customers ÷ attended appointments. This step depends partly on the sales team, though weak leads show up here too.

Add one more number alongside the funnel: speed to lead, the time between a visitor's first message and the first useful reply. A reply within seconds is one of the main reasons businesses use AI lead capture software, so a slow speed to lead usually points to a setup or integration problem.

Voice Agent KPIs

Phone calls bring problems that chat metrics miss. These measures cover them:

  • Response delay: the pause between a caller finishing a sentence and the agent replying. Long silences make callers talk over the agent or hang up.

  • Interruption handling: whether the agent stops speaking when the caller cuts in. Listen to a handful of calls to check this.

  • Transfer rate: the share of calls sent to a person. Read it alongside the reason for each transfer.

  • Call abandonment rate: the share of callers who hang up before their issue is handled. This is the guard metric for voice.

  • Average handle time: how long calls last. Shorter is better only when the issue still gets resolved.

  • Repeat rate: how often callers have to say the same thing twice, often a sign of poor speech recognition or unclear questions.

Teams using AI voice agent software for inbound calls should review call recordings, not just transcripts. Tone, pauses and crosstalk only show up in the audio.

Cost Per Resolution

Formula: total monthly AI cost ÷ automated resolutions

Take a business with 1,000 conversations a month and a 60% resolution rate, or 600 resolved conversations. If the AI costs $300 that month, each resolution costs $0.50.

To compare that with people, work out your own cost per conversation. A team member paid $30 an hour who handles 10 chats an hour costs $3 per chat. On those numbers, the AI handles routine questions at a sixth of the cost.

If the reopen rate is high, divide the AI cost by resolutions that stayed resolved instead. That gives the true figure, and it is often noticeably higher.

KPI Benchmarks at a Glance

The ranges below come from Chaindesk's 2026 chatbot benchmarks. They are vendor figures without independent studies behind them, so treat them as starting points rather than fixed targets.

Custom HTML/CSS/JavaScript

Industry changes the picture. Healthcare and finance agents usually hand off more often by design, because some questions should always reach a person.

How to Review Conversations When Volume Is Low

Large companies use AI tools to score every conversation. A small business or a new agency client rarely needs that. A weekly sample works well:

  1. Pick 25 conversations at random from the past week.

  2. Score each one yes or no on four checks: accurate, resolved, handed off when it should have been, and right tone.

  3. Tag every "no" with its cause: missing knowledge, unclear instructions or a broken integration. Unclear instructions are often fixed fastest by starting from proven AI customer service prompt templates.

  4. Fix the most common cause first, then pull a fresh sample next week.

Twenty-five conversations take under an hour to review, and patterns show up within two or three weeks.

A Simple Monthly KPI Report

A useful report fits on one page. These six lines cover what an owner or client needs to know:

  1. Main KPI for each agent job: this month's number

  2. Guard metric for each: this month's number

  3. Change from last month: up, down or flat, with a one-line reason

  4. Top three unanswered questions: and whether they have been fixed

  5. Cost per resolution: this month's figure

  6. One action for next month: a single, specific change

Agencies can send the same report to every client each month, which makes progress easy to compare. Those using white-label AI agent software can deliver it under their own brand from the same dashboard their clients log into.

Frequently Asked Questions

What is a good resolution rate for an AI agent?

It depends on the business and the type of questions. A store answering order and return questions can reach a high rate quickly, while a business with complex or custom requests will settle lower. Your own trend over three to six months matters more than any industry number.

How often should I review AI agent KPIs?

Check the main numbers weekly during the first month, then monthly once they settle. Review sooner after any big change, such as a new product line or policy.

Should I trust the resolution rate my vendor reports?

Use it, but ask three questions first: how a resolution is defined, whether abandoned chats count as resolved, and whether answers are checked for accuracy. Clear answers make the number far more useful.

What tools do I need to track these KPIs?

Most AI agent platforms show conversations, handoffs and unanswered questions out of the box. The lead funnel often needs a CRM or a simple spreadsheet that joins chat data with bookings and sales.

How do I compare AI and human agents fairly?

Score both on the same checklist, and compare them on similar question types. AI usually handles the simple questions, so comparing its numbers with staff who get the hardest cases will mislead.

How long until the numbers settle?

Most agents need four to eight weeks of real conversations before the trends become reliable. Early weeks swing a lot as content gaps get filled.

Final Thoughts

Before the first report goes out, write down a target for each main KPI and agree on it with whoever reads the report. Measuring against a target set in advance keeps the review honest, because a number can only be good or bad compared with what you expected it to be.

Stammer.ai

Stammer.ai

Posted by the Stammer.ai team.

Instagram logo icon
Youtube logo icon
Back to Blog