Observability and evaluation for voice agents
Voice agents fail in ways a normal APM cannot see: the caller waited nine seconds, the transcription dropped a word that changed the booking, the model was retried twice inside one turn. Vaanieval captures enabled audio, transcripts, provider spans and timing evidence to help you investigate a call — with gaps made visible, not filled in.
Python and Node.js SDKs · local-first capture · explicit dashboard upload · optional external evaluation
Hosted beta is under construction. GitHub repositories are public; these guides use public Git installs. No npm or PyPI release was found under the documented package names in the 20 September 2026 registry checks. Email shubham@vaanieval.com for setup help, or book a call so we can understand your use case and get you set up.
Synthetic localhost fixture: one tool span and one second of silence. No conversation, measured voice latency, customer data or provider calls. This is not a hosted-service capture.
Find the slow turn, not the slow service
Vaanieval groups captured STT, LLM, tool and TTS spans by turn so you can investigate latency and retries. Timing coverage depends on your instrumentation; missing evidence is shown as unavailable.
Listen to the captured call
When audio capture is enabled and frames are supplied, review timeline-aligned stereo PCM with the agent on the left and caller on the right. Transcript availability depends on capture settings and integration coverage.
Check the transcript, not just the timing
Optionally compare a recorded caller track with ElevenLabs Scribe v2 and use OpenAI transcript judging to prioritise disagreements for human review. Either model can be wrong; disagreement is not ground-truth accuracy.
Start with a local recorded call
Install the SDK, point it at a localhost dashboard, and open a synthetic call. Hosted beta work is separate and not a public SaaS release. Follow the public Git-based instructions. Public source availability does not establish a cloud-compatible SDK release or hosted-service readiness.