Pick your conversation

A phone on a table that separates several conversations happening at once, to help people with hearing loss follow the one they choose. I own speaker diarization and separation.

Group project course, Linköping University, in progress2026

Diagram of the audio pipeline from one phone, with diarize, match, extract and gate marked as my part
Diagram of the audio pipeline from one phone, with diarize, match, extract and gate marked as my part.

In a busy room, several people talk at the same time. This group project puts one phone on the table to work out who is speaking, separate the voices, and transcribe the conversations for hearing-impaired listeners.

My part

My teammates handle capture, transcription, conversation tracking and the interface. I own two steps:

  • Speaker diarization: working out who spoke when.
  • Speaker separation: extracting the voice of an enrolled speaker from the mix.

The approach so far

I fine-tune pretrained models instead of training from scratch, starting from pyannote. The pipeline runs in order: diarize, match voices to enrolled names, extract each speaker, then gate. It starts offline, measured with DER and SI-SDR. Streaming comes later.

This project is still running, so there are no final results yet.

More work