Milestones and samples

An operation's duration tells you how long a provider took. It does not tell you what the caller experienced. Milestones close that gap.

The problem with durations

Consider a turn where transcription took 3.3 seconds. Two very different things could have happened:

1. The recognizer produced text quickly but took 2.5 s to decide the caller had stopped talking.
2. The recognizer decided quickly but took 2.5 s to produce final text.

Same duration. Different team, different fix. A milestone timeline separates them:

The caller-visible reply wait is caller stops speaking → first audio byte. Everything else explains it.

Recording a milestone

op.event("first_token")
op.event("first_partial", {"text_length": 12})
op.event("first_byte", occurred_at_ms=session.now())

If you do not pass occurred_at_ms / occurredAtMs, the SDK stamps it with the session clock at the moment of the call — so record milestones at the moment they happen, not in a batch at the end.

Repeats accumulate

Calling event() twice with the same name does not overwrite. It merges:

{
  "received_frame": {
    "occurred_at_ms": 1240,
    "last_at_ms": 8830,
    "count": 214,
    "total_bytes": 918_400
  }
}

You keep the first occurrence, the most recent one, and how many there were, for a fixed cost. That is what makes it safe to call event() from a websocket frame handler running hundreds of times per call.

Samples

sample() retains a bounded series of low-frequency observations. Partial transcripts are the motivating case: worth keeping for a latency timeline, but an unbounded list would turn an audio stream into an unbounded event stream.

op.sample("partial_transcript", {"text": "how do i change"}, limit=100)

At the limit (default 100 per name), further samples are dropped and the bucket is marked truncated: true. The truncation is visible in the package rather than silent.

The vocabulary the dashboard reads

These are the milestone names the demo surfaces. Emit them with these names and the console labels your turns automatically; emit your own and they still appear in the trace, just without the derived latency.

STT

LLM

TTS

Websocket connection spans

Missing milestones are unmeasurable, not zero

This is the single most important convention in the product.

If a turn is missing any milestone in the chain the dashboard needs, that turn is excluded from the percentile and counted in the denominator as unmeasurable. It is never:

• estimated from the operation's start and end times,
• inferred from a batch HTTP round trip,
• or rendered as 0.

That is why the STT review header shows ENDPOINTING — unavailable rather than 0.00s, and why the fleet cards publish "68% of 111 turns measurable" alongside every percentile.

A worked example

turn = session.start_turn()

stt = turn.start_operation(type="stt", provider="deepgram", model="nova-3")
stt.event("speech_started")
# ... streaming frames arrive
stt.sample("partial_transcript", {"text": partial})
stt.event("first_partial")
stt.event("speech_ended")
stt.event("speech_final")
stt.event("final_transcript")
stt.end(status="ok", response={"transcript": text})

llm = turn.start_operation(type="llm", provider="openai", model="gpt-4o-mini")
llm.event("request_body_captured")
llm.event("first_token")
llm.end(status="ok", response={"tokens": 128})

tts = turn.start_operation(type="tts", provider="elevenlabs")
tts.event("speak")
tts.event("first_byte")
tts.end(status="ok")

turn.end()

That turn is fully measurable: every card in the dashboard has what it needs, and the waveform's silence annotation can name which stage owned the gap.

Next