Tour the live demo

demo.vaanieval.com is a frozen, read-only snapshot of 30 real voice-agent calls from August 2026. The audio, transcripts, timings and traces are genuine captures, anonymised. Nothing you click changes anything.

It is the fastest way to decide whether VaaniEval measures what you care about.

Start with one call

The call view is the heart of the product: everything else is a way of finding which call to open.

Four things to look at, in order:

Expand a turn

This is where the real answer usually is.

Two details in that screenshot are worth the whole tour:

• Milestones, not just durations. An STT span records caller starts speaking → first partial transcript → caller stops speaking → provider marks speech final → final transcript → end of utterance. That is enough to separate "the recognizer was slow to produce text" from "the recognizer was slow to decide the caller had finished" — two different problems with two different fixes.

• The framework span versus the HTTP attempts. Turn 3's LLM operation reports 19.9 s and carries the note "2 HTTP attempts served this one model call, 1 of which failed and was retried. The framework span above times the whole sequence, which is why its duration covers every attempt." A request-level APM would have shown you two requests of 10.7 s and 8.9 s and no reason the caller waited 24 seconds.

There is also a cancelled LLM attempt annotated: "stopped before it finished and no caller speech overlaps it, so this is not barge-in." An operation that stopped because something cancelled it is not automatically a fault — a TTS span aborted by barge-in is the agent behaving correctly — so the product distinguishes the two rather than counting both as failures.

Find the call worth opening

The Dashboard tab rolls the same measurements across a time range, agent, provider, model, SDK, environment and version.

Note what the KPI cards do when the sample is thin: "still a small sample", "not enough history to compare", "68% of 111 turns measurable". The denominator is always published, because a p95 over 75 measured turns is a very different claim from a p95 over 75,000.

Calls needing attention ranks in three classes — failed (the caller heard an error), unverifiable (capture is incomplete, so we cannot say what they heard) and slow — then by magnitude within each class. Opening a row lands on the turn that caused the flag, not the top of the call.

Get told without looking

The Alerts tab evaluates rules against the same calls the dashboard reports on.

Each rule shows its scope (3 of 4 agents, 1 no reading), the reading against its threshold, and the destination. The no reading count matters: a rule that cannot evaluate on an agent says so rather than quietly reporting healthy.

Review the transcription

The STT review tab on a call shows streaming timing and transcripts captured from production.

Two conventions to internalise:

1. — means unavailable, and it always carries a reason. In the screenshot ENDPOINTING is — unavailable. It is never rendered as 0, and never inferred from an operation's start and end times or from a batch HTTP round trip.
2. Accuracy only appears after a challenger comparison. The banner says so explicitly: "Streaming timing and transcripts. Accuracy appears after a challenger comparison." Until you replay the call against a challenger model, there is nothing to compare the production transcript to.

Running a comparison produces an estimated WER, per-word substitutions, deletions and insertions, and a judge pass that flags only the disagreements that could have changed the conversation. See STT evaluation and Metrics.

Next