mlsanityData to 14 Oct 2026

Categorize Early, Integrate Late: Divergent Processing Strategies in Automatic Speech Recognition

arXiv 2601.06972 ↗First posted 11 Jan 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

In speech language modeling, two architectures dominate the frontier: the Transformer and the Conformer. However, it remains unknown whether their comparable performance stems from convergent processing strategies or distinct architectural inductive biases. We introduce Architectural Fingerprinting, a probing framework that isolates the effect of architecture on representation, and apply it to a controlled suite of 24 pre-trained encoders (39M-3.3B parameters). Our analysis reveals divergent hierarchies: Conformers implement a "Categorize Early" strategy, resolving phoneme categories 29% earlier in depth and speaker gender by 16% depth. In contrast, Transformers "Integrate Late," deferring phoneme, accent, and duration encoding to deep layers (49-57%). These fingerprints suggest design heuristics: Conformers' front-loaded categorization may benefit low-latency streaming, while Transformers' deep integration may favor tasks requiring rich context and cross-utterance normalization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 33 references that are in the map. First posted on arXiv 11 Jan 2026.

Builds on

14 of 33 references.

References · 14

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
  5. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  6. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  7. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 92
  8. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 91
  9. 2017Representations of language in a model of visually grounded speech signalcited by 37
  10. 2023Probing self-supervised speech models for phonetic and phonemic information: a case study in aspirationcited by 36
  11. 2017Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systemscited by 32
  12. 2019Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognitioncited by 16
2 more
  1. 2021Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representationscited by 16
  2. 2020Neural Representations for Modeling Variation in Speechcited by 14

Built on by

Not cited yet.