GitHub - Kayvan-Zahiri/asr-age-gap: Speech recognition does not degrade with speaker age. Voice-agent turn-taking does, by 2-5x.

Voice agents are being pointed at elderly callers, and the assumed risk is that speech recognition will not hear them. That assumption is wrong, and it is hiding the failure that is actually happening. Measured on 2,760 Common Voice clips, matched between age brackets on accent, gender and speaker so the only thing varying is age, and checked against a second 3,189-clip draw that controls for none of it: word error rate premature cutoff (700 ms) twenties n=920 6.53% 8.0% sixties n=920 5.23% -1.31pp * 19.7% +11.6pp * seventies n=920 4.67% -1.86pp * 16.6% +8.5pp * * speaker-bootstrapped 95% interval excludes zero Whisper transcribes older speakers more accurately, not less. And where a stack endpoints on a fixed silence threshold, those same speakers get talked over two to two and a half times as often. That second finding has a caveat that arrived after publication and is worth reading before quoting the number: a semantic turn model closes most of the gap. Measured against Pipecat's smart-turn v3 on the same clips, +11.6pp becomes +5.9pp and stops excluding zero (section 3). The fixed-threshold result describes a real and common configuration, not every configuration. python3 bench/run.py reproduces both. No API key, no spend. 1. The recognition penalty is not there Large-v3, wideband, speaker-level 95% intervals: bracket n spk WER vs twenties twenties 920 606 6.53% [5.77, 7.40] baseline sixties 920 240 5.23% [4.65, 5.86] -1.31pp [-2.33, -0.30] excludes zero seventies 920 116 4.67% [4.12, 5.31] -1.86pp [-2.91, -0.88] excludes zero Every error type falls with age, so this is not one category masking another: bracket sub del ins twenties 5.22% 0.57% 0.74% sixties 4.19% 0.41% 0.64% seventies 3.67% 0.41% 0.59% Deletions in particular do not rise, which is the result you would expect if quiet or breathy speech were being dropped. It is not being dropped. It is not a Whisper artifact. The obvious objection is that Whisper's decoder is a language model, so it might be repairing older speakers' word choices rather than hearing them better. So the same clips were re-run through wav2vec2, which is pure CTC: frame-wise, greedy, no decoder and no implicit LM. bracket Whisper enc-dec wav2vec2 CTC twenties 6.53% 14.23% sixties 5.23% -1.31pp 10.30% -3.94pp [-5.52,-2.40] seventies 4.67% -1.86pp 10.52% -3.72pp [-5.49,-2.08] Absolute WER is much higher for wav2vec2 (LibriSpeech-only training, no LM), so only the between-bracket comparison transfers. The effect is larger there and still excludes zero, which puts it in the acoustics rather than in a decoder. 2. The turn-taking penalty is large A voice agent decides the caller has finished by waiting for a fixed stretch of silence. A pause inside an utterance that exceeds that threshold is heard as the end of the turn, and the agent starts talking over someone mid-sentence. WER is blind to this. The words the model did receive can be transcribed perfectly while the caller is cut off every time. Share of utterances containing an internal pause at least this long: bracket 400 ms 500 ms 700 ms 1000 ms twenties 23.3% 17.3% 8.0% 2.0% sixties 41.5% 33.7% 19.7% 8.7% seventies 37.3% 28.6% 16.6% 6.2% All six age-versus-baseline differences exclude zero. At 700 ms the gap is +11.6pp [+7.7, +15.7] for the sixties and +8.5pp [+4.0, +13.6] for the seventies. The mechanism is in the timing. Older speakers take twice as many internal pauses and spend twice as long in them: bracket words/voiced-s pauses/clip pause total twenties 2.47 1.0 240 ms sixties 2.20 2.0 480 ms seventies 2.20 2.0 420 ms It is not a clean gradient. The sixties are cut off slightly more than the seventies and their intervals overlap. This reads as an effect that arrives by 60 and plateaus, not a straight line, and it is drawn that way rather than smoothed. The eighties, run separately because matching against them would have shrunk every bracket eightfold, are the sharpest case. Only 14 speakers exist, so the intervals are wide — and the effect clears them anyway: threshold twenties eighties difference [95% CI] 400ms 27.9% 49.2% +21.8% [+6.3%, +36.6%] 500ms 15.6% 40.2% +24.8% [+8.9%, +38.7%] 700ms 4.1% 22.1% +18.1% [+7.9%, +27.2%] 1000ms 0.0% 6.6% +6.5% [+1.7%, +14.5%] At 700 ms that is a 5.4x gap. Their WER, meanwhile, is 6.24% against 6.01% — a difference of +0.15pp whose interval comfortably includes zero. The two findings diverge further with age: recognition stays flat while turn-taking gets steadily worse. The result survives a re-draw. Running the whole benchmark again without accent matching — 3,189 clips, 1,434 speakers, 30 accents instead of 8 — reproduces the cutoff rates almost exactly: twenties sixties seventies matched 8.0% 19.7% 16.6% unmatched 7.5% 19.3% 16.6% This is a re-draw from one corpus, not an independent replication: the two samples share 52% of their speakers, though only 18% of their clips. It shows the numbers are not an artifact of one particular draw or of the accent matching. It does not show they generalise beyond Common Voice. 3. A semantic turn model closes most of the gap Mark Backman of Daily/Pipecat read an earlier version of this and pointed out that it described the wrong thing: production stacks do not endpoint on a fixed VAD threshold. Pipecat's default is smart-turn, a semantic model that listens to the waveform and grants more time when the turn sounds unfinished. He is right, so smart-turn v3 was measured on the identical sample. For each clip, the audio up to an internal pause is fed to the model and it is asked whether the turn is complete. The speaker demonstrably continues, so "complete" is a false cutoff. fixed 700ms threshold smart-turn v3 twenties 8.0% 75.6% sixties 19.7% +11.6pp * 81.6% +5.9pp [-0.9,+12.9] seventies 16.6% +8.5pp * 79.7% +4.0pp [-3.6,+11.6] The gap roughly halves and stops excluding zero. Positive control on whole utterances is flat at 90-91% across brackets. Two things this does not say. The absolute 76-82% rate is not an error rate: many internal pauses are legitimate clause boundaries where a turn could plausibly end, and without human labels on which prefixes sound complete, only the between-bracket comparison is interpretable. And "includes zero" is not "no effect" — both point estimates stay positive, and 86 seventies speakers cannot resolve four points either way. The practical reading: if you endpoint on a fixed threshold, the age gap is real and large. If you use a semantic turn model, most of it goes away. The published smart-turn benchmark stratifies 31,527 samples across 23 languages but not by speaker age, and its training mix leans on synthetic TTS, which does not pause the way an eighty-year-old does. 4. What a person's own speech noise costs a drift detector Several products now offer daily phone check-ins for older adults that claim to flag cognitive decline from voice biomarkers. Validating that needs gated clinical corpora. But a prior question needs no clinical labels and bounds the claim from below: how much does one healthy person's speech vary between utterances? A drift detector can only see change that clears the speaker's own noise. Measured on 36 speakers with 40+ clips each: feature within-speaker CV speech rate ~18% utterance duration ~23% number of internal pauses ~76% total pause time ~96-111% Converted to the sample needed to resolve a 10% change at 80% power: feature utterances calls @40/call speech rate 25 0.6 utterance duration 41 1.0 number of internal pauses 447 11 total pause time 758 19 Pause features — the most frequently cited voice biomarker — vary by about 100% within the same speaker, often within one sitting. Detecting a 10% shift in total pause time takes roughly three weeks of daily calls per reading, so a "six-week trend" is two or three noisy measurements. Speech rate and duration are comfortably usable. Both directions of error are stated: Common Voice clips from one contributor are often a single sitting, so real day-to-day variance is larger; and utterances within one call are correlated, so dividing by 40 overstates the effective sample. Both push the true requirement up. These are floors. 5. The accent confound is real, and it does not drive the result Common Voice is globally crowdsourced and its younger contributors skew non-native. The twenties bracket is 11.9% India-and-South-Asia English and 46.9% native anglophone; the sixties are 64.4%. Whisper is worse on non-native English, so age and accent are genuinely entangled in this corpus, and an uncontrolled comparison has an obvious alternative explanation. Brackets are therefore matched on the (accent, gender) pair, holding both identical by construction: 920 clips per bracket, 8 accents, 392/528 male/female in every bracket. I expected that to change the answer. It does not. Running the benchmark both ways, on samples whose accent composition could hardly be more different: matched unmatched (8 accents, (30/15/13 accents, identical mix) differing mix) twenties 6.53% 6.60% sixties 5.23% 4.90% seventies 4.67% 5.11%