mlsanityData to 7 Oct 2026

A Probe Shift Is Not a Fairness Fix: The Limits of Representation Steering in Speech Models

arXiv 2609.18533 ↗First posted 16 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Automatic speech recognition (ASR) systems exhibit unequal error rates across speaker groups, motivating interventions on their internal representations. We ask whether speaker-linked attributes that are linearly readable from pretrained ASR encoders yield useful directions for reducing group word-error-rate (WER) gaps. Across Whisper-medium, HuBERT-large, and Wav2Vec2-large on Common Voice and the Speech Accent Archive, we probe every encoder layer for metadata-derived sex/gender, age, and native/accent labels; construct centroid and probe-derived directions; inject them at selected layers; and compare downstream probe trajectories with matched WER changes. Sex labels are highly decodable (best macro-F1 0.924--0.941), native/accent labels are also above chance (0.544--0.696), and age is weaker (0.354--0.397). Of 22 post-selected reruns, nine have 95% paired-bootstrap intervals entirely below zero, yet every absolute source-group WER reduction is below 0.7 percentage points. Conversely, a local target-class probe rate can rise from 8.09% to 99.87% while WER worsens. Linear readability is therefore neither evidence of causal use nor a reliable mitigation method. Our results motivate evaluating speech-bias interventions jointly at representation, propagation, and task levels.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 26 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

19 of 26 references.

References · 19

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  5. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  7. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  9. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  10. 2021Quantifying Bias in Automatic Speech Recognitioncited by 50
  11. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 40
  12. 2017Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systemscited by 31
7 more
  1. 2025To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Modelscited by 26
  2. 2024Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gapscited by 24
  3. 2024On the social bias of speech self-supervised modelscited by 16
  4. 2020Adversarial Disentanglement of Speaker Representation for Attribute-Driven Privacy Preservationcited by 13
  5. 2020Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASRcited by 11
  6. 2026Identifying and typifying demographic unfairness in phoneme-level embeddings of self-supervised speech recognition modelscited by 1
  7. 2026Speaker Group Encoding in Self-supervised Speech Recognition Modelscited by 1

Built on by

Not cited yet.