Speaker Diarization Explained: How to Identify Who Said What

What is speaker diarization?

Speaker diarization is the process of partitioning an audio recording by who is speaking. The word comes from "diary", in the sense of keeping a record of who did what and when. In practice, diarization takes a single audio stream that may contain several people and divides it into segments, labelling each segment with a distinct speaker: Speaker 1, Speaker 2, Speaker 3, and so on. It answers a deceptively simple question: who spoke when?

Crucially, diarization on its own doesn't tell you what was said, and it doesn't tell you the real name of each speaker. It only groups the audio by voice. To turn that into a readable "who said what" transcript, diarization is combined with high-accuracy speech recognition: the recognition converts speech to text, and the diarization attributes each stretch of text to a speaker. Australian Transcription runs both together, so a single job returns both the plain transcript and the speaker-labelled segments.

Diarization vs transcription vs voice recognition

These three terms are often used interchangeably, but they describe genuinely different tasks. Getting them straight matters, because it changes what output you can expect and what the technology can and can't do.

Transcription (speech recognition) converts spoken words into written text. Its job is accuracy of words, not attribution of speakers. On its own, a raw transcript is one continuous block of text with no indication of speaker changes.

Speaker diarization partitions the audio by distinct voices and labels the segments anonymously. It knows there are, say, two speakers and which parts belong to each, but it does not know their names. Speaker 1 might be the interviewer and Speaker 2 the interviewee, but the system is only saying "these are two different people".

Voice recognition (speaker identification) goes one step further and matches a voice to a specific, known person. This requires enrolment: a voiceprint of each individual is captured in advance, and the system compares incoming audio against those enrolled profiles to output a real name. This is the technology behind "Hi, it's me" voice authentication, and it is a different, more invasive capability with its own privacy considerations.

Australian Transcription performs diarization, not voice recognition. It will tell you there are two distinct speakers and which segments belong to each, but it will never claim a segment was spoken by a named individual. You attach the real names yourself, from context, once you have the labelled segments.

Common use cases

Almost any recording with more than one person benefits from diarization. A few of the most common:

• Meetings: Zoom, Teams, and in-person meeting recordings become far more useful when each contribution is attributed to a speaker, so action items can be traced back to who committed to them.
• Interviews: journalists and researchers need to separate the interviewer's questions from the interviewee's answers, cleanly and automatically.
• Medical consults: separating the clinician from the patient makes a consultation transcript usable for notes and record-keeping.
• Legal: depositions, hearings, and recorded statements rely on knowing exactly who said each line.
• Call centres: splitting the agent from the customer supports quality assurance, compliance review, and analytics.
• Podcasts: multi-host and guest episodes can be turned into accurate, speaker-attributed show notes and transcripts.

Why diarization is hard

Diarization sounds simple, but it is one of the harder problems in speech processing. A human listener effortlessly tracks who's talking; getting a machine to do it reliably runs into several real obstacles.

• Overlapping speech: when two people talk at once, the audio contains both voices in the same instant, and deciding where one speaker ends and the other begins is genuinely ambiguous.
• Similar voices: two speakers of the same gender, age, and accent can have voiceprints that are close enough to be easily confused, causing the system to merge them or swap their labels.
• Crosstalk and background noise: side conversations, room echo, and low-quality microphones blur the boundaries between speakers and add voices that aren't the main participants.
• Unknown speaker count: if you don't tell the system how many people are present, it has to infer it, and estimating the number of speakers is itself an error-prone step that affects everything downstream.

That last point is why the API lets you supply the expected number of speakers. When you know how many people are in the recording, providing that count removes one of the hardest parts of the problem and produces noticeably cleaner segments.

How to get speaker labels with the Australian Transcription API

The API is asynchronous: you submit an audio file with the number of speakers you expect, receive a job_id, then poll until the job is complete. When it finishes, the response includes a diarization field: a list of segments, each with speaker, start, end, and text. Set num_speakers to an integer between 1 and 10 (default 2). Diarization is included at no extra cost in the standard $0.02 AUD per minute price.

import time
import requests

API_KEY = "your_api_key_here"
BASE_URL = "https://api.icana.ai/api/v1"
HEADERS = {"X-API-Key": API_KEY}


def diarize(audio_path: str, num_speakers: int = 2) -> list:
    """Submit an audio file and return speaker-labelled segments."""

    # Step 1: Submit the file, telling the API how many speakers to expect
    with open(audio_path, "rb") as f:
        response = requests.post(
            f"{BASE_URL}/transcribe",
            headers=HEADERS,
            files={"file": f},
            data={"num_speakers": 2},
        )
    response.raise_for_status()
    job_id = response.json()["job_id"]
    print(f"Job submitted: {job_id}")

    # Step 2: Poll until the job is complete (max 60 attempts, ~5 min)
    for attempt in range(60):
        result = requests.get(
            f"{BASE_URL}/jobs/{job_id}",
            headers=HEADERS,
        )
        result.raise_for_status()
        data = result.json()

        status = data["status"]
        print(f"Status: {status}")

        if status == "complete":
            # diarization is a list of speaker-labelled segments
            return data.get("diarization", [])
        elif status == "failed":
            raise RuntimeError(f"Transcription failed: {data.get('error')}")

        time.sleep(5)

    raise TimeoutError(f"Job {job_id} did not complete within 5 minutes")


if __name__ == "__main__":
    segments = diarize("meeting.mp3", num_speakers=2)

    # Each segment has: speaker, start, end, text
    for segment in segments:
        speaker = segment["speaker"]
        start = segment["start"]
        end = segment["end"]
        text = segment["text"]

        # Format: [Speaker 1 | 0:00 - 0:12] Thanks everyone for joining.
        start_fmt = f"{int(start // 60)}:{int(start % 60):02d}"
        end_fmt = f"{int(end // 60)}:{int(end % 60):02d}"
        print(f"[{speaker} | {start_fmt} - {end_fmt}] {text}")

The diarization list gives you the "who said what" view, while the separate transcription field on the same response gives you the plain, unsegmented transcript. Remember that the speaker labels are anonymous (Speaker 1, Speaker 2, …); you map them to real names yourself from context. All of this runs on AWS infrastructure in Sydney (ap-southeast-2), so your audio never leaves Australia and APP 8 cross-border disclosure obligations are never triggered.

Frequently asked questions