How to Transcribe Audio with Timestamps

Segment-level, not word-level

Being precise about this up front saves disappointment. Our API returns segment-level timestamps: a start and end time for each contiguous chunk of speech, which usually corresponds to a sentence or a speaker turn. It does not return a timestamp for every individual word.

That distinction matters for exactly one use case: word-by-word highlighting as audio plays, karaoke style. If that is what you are building, you need a service with word-level output and we are not it.

For everything else — jumping to a passage, generating captions, building chapters, citing a quote with a timecode, measuring speaking time per participant — segment-level timing is what you actually want, and word-level would only add noise.

Getting timestamps

There is nothing to enable. Every completed job returns two fields: transcription, which is plain text with no timing, and diarization, which carries the timing and the speaker labels.

SPEAKER_00 [00:00:01.000 - 00:00:05.240]: Thanks for coming in today.
SPEAKER_01 [00:00:05.900 - 00:00:11.480]: No problem at all, happy to help.
SPEAKER_00 [00:00:12.010 - 00:00:19.750]: Let's start with how you first heard about us.

Times are HH:MM:SS.mmm from the start of the recording, with millisecond precision. Speaker labels are generic — SPEAKER_00, SPEAKER_01 — and map to real people in your own code.

Parsing into something usable

import re

LINE = re.compile(
    r"^(?P<speaker>SPEAKER_\d+) "
    r"\[(?P<start>[\d:.]+) - (?P<end>[\d:.]+)\]: "
    r"(?P<text>.*)$"
)


def to_seconds(timecode: str) -> float:
    hours, minutes, seconds = timecode.split(":")
    return int(hours) * 3600 + int(minutes) * 60 + float(seconds)


def parse(diarization: str) -> list[dict]:
    segments = []
    for line in diarization.splitlines():
        match = LINE.match(line.strip())
        if not match:
            continue
        seg = match.groupdict()
        seg["start_seconds"] = to_seconds(seg["start"])
        seg["end_seconds"] = to_seconds(seg["end"])
        seg["duration"] = seg["end_seconds"] - seg["start_seconds"]
        segments.append(seg)
    return segments

Converting to seconds immediately is worth doing, because almost everything downstream — media player seeking, arithmetic, sorting — wants a number rather than a string.

Clickable transcripts

The highest-value thing to build with timestamps is a transcript where clicking a line seeks the audio to that moment. It takes very little code and transforms how usable a long recording is.

def render_html(segments: list[dict], audio_url: str) -> str:
    rows = "\n".join(
        f'<p data-start="{s["start_seconds"]:.3f}" class="cue">'
        f'<span class="time">{s["start"][:8]}</span> '
        f'<strong>{s["speaker"]}</strong>: {s["text"]}</p>'
        for s in segments
    )
    return f"""
<audio id="player" src="{audio_url}" controls></audio>
<div id="transcript">{rows}</div>
"""

const player = document.getElementById("player");

document.getElementById("transcript").addEventListener("click", (event) => {
  const cue = event.target.closest(".cue");
  if (cue) {
    player.currentTime = Number(cue.dataset.start);
    player.play();
  }
});

// Optional: highlight the line currently being spoken.
const cues = [...document.querySelectorAll(".cue")];
player.addEventListener("timeupdate", () => {
  const now = player.currentTime;
  let active = null;
  for (const cue of cues) {
    if (Number(cue.dataset.start) <= now) active = cue;
    else break;
  }
  cues.forEach((c) => c.classList.toggle("active", c === active));
});

Chapters and speaking time

Two more things fall out of segment timing almost for free.

from collections import defaultdict

def speaking_time(segments: list[dict]) -> dict[str, float]:
    totals = defaultdict(float)
    for seg in segments:
        totals[seg["speaker"]] += seg["duration"]
    return dict(totals)


# Useful for interview QA: did the interviewer talk too much?
totals = speaking_time(segments)
total = sum(totals.values())
for speaker, seconds in sorted(totals.items()):
    print(f"{speaker}: {seconds / 60:.1f} min ({seconds / total:.0%})")

Researchers and UX teams use this as a quality check on interview technique. A moderator taking 60% of the airtime in a discovery interview is a finding in itself.

def chapters(segments: list[dict], gap_seconds: float = 3.0) -> list[dict]:
    """Start a new chapter after a noticeable silence."""
    marks, current = [], None
    for i, seg in enumerate(segments):
        gap = seg["start_seconds"] - segments[i - 1]["end_seconds"] if i else 0
        if current is None or gap >= gap_seconds:
            current = {"start": seg["start_seconds"], "title": seg["text"][:60]}
            marks.append(current)
    return marks

Pause-based chaptering is crude but surprisingly effective on interviews and presentations, where topic changes genuinely do follow a beat of silence. Tune the gap threshold to your material.

Accuracy of the timing itself

Segment boundaries are derived from the diarization model's view of where speech starts and stops, so they are accurate to a fraction of a second in clean audio and less precise in messy audio.

Where they drift is crosstalk. When two people speak over each other, the boundary between their segments is a judgement the model makes, and it can place a word on the wrong side. For navigation this is immaterial — you land within a second of the right place. For captions it can produce a slightly early or late cue.

If exact boundaries matter, offset your seek target back by half a second. Landing slightly early is unnoticeable; landing slightly late means missing the first word of the quote you were looking for.