The signal chain: how sound is prepared for recognition

Before Whisper, each speech fragment goes through ffmpeg:

-af "highpass=f=80,lowpass=f=6000,afftdn=nf=-20,acompressor=threshold=-25dB:ratio=4:attack=5:release=50,loudnorm" \
-ac 1 -ar 16000

• highpass=80 cuts hum, vibration and excess low end
• lowpass=6000 removes high-frequency hiss
• afftdn suppresses constant background noise
• acompressor evens out quiet and loud phrases
• ratio=4 compresses peaks at roughly 4:1
• attack=5 reacts quickly to sharp sounds
• release=50 brings the level back smoothly
• loudnorm normalises the final loudness
• mono, 16 kHz bring the recording to Whisper’s format

It’s easy to overdo it here.

Heavy noise reduction starts removing word endings along with the background. Compression can raise the noise, and a noise gate can cut off quiet speech.

So the main mode cleans the recording but tries not to damage the voice.

For difficult conditions there’s a harsher Noisy preset.

As a result Whisper receives not the original iPhone recording but a prepared signal with more even loudness and less background noise.