How LiveContext separates my voice from the other person's

For LiveContext to understand a conversation, it first has to solve a fairly basic problem: who exactly is speaking right now?

I could have recorded the whole call as one audio stream and then tried to work out in software which voice was mine and which was the other person’s.

In LiveContext I went another way: I separate the voices before transcription.

That gives two independent sources:

my microphone → YOU

call audio → REMOTE

YOU comes straight from the physical microphone: AirPods or a USB mic.

REMOTE is more interesting. Sound from an app, such as Telemost or a browser, goes into a separate virtual audio device, LiveContext Remote, which I built with Loopback.

So the computer effectively gets two different audio tracks:

microphone ───────────→ YOU

→ LiveContext
/
Telemost / browser ─────→ REMOTE

Each stream has its own processing, its own speech detection settings and its own queue. Only after that does the sound go to local transcription.

As a result, LiveContext doesn’t have to guess who a given phrase belongs to.

It knows that from the sound source itself.

This small architectural decision turned out to matter for everything built on top of it. Because for the system there’s a big difference between:

“the client said the price is high”

and

“I told the client the price is high”.

The text is almost identical. The state of the conversation is completely different.

Then comes the harder part: how to turn these two continuous streams of sound into fast and accurate local transcription without bringing the whole computer to its knees.