Probing Statistical Representations For End-To-End ASR
Not on the map. Why?
Abstract (arXiv)
End-to-End automatic speech recognition (ASR) models aim to learn a generalised speech representation to perform recognition. In this domain there is little research to analyse internal representation dependencies and their relationship to modelling approaches. This paper investigates cross-domain language model dependencies within transformer architectures using SVCCA and uses these insights to exploit modelling approaches. It was found that specific neural representations within the transformer layers exhibit correlated behaviour which impacts recognition performance. Altogether, this work provides analysis of the modelling approaches affecting contextual dependencies and ASR performance, and can be used to create or adapt better performing End-to-End ASR models and also for downstream tasks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 26 references that are in the map. First posted on arXiv 3 Nov 2022.
Builds on
16 of 26 references.
References · 16
- 2017Attention Is All You Needcited by 59,932
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
- 2017A Closer Look at Memorization in Deep Networkscited by 936
- 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
- 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
- 2014On the Number of Linear Regions of Deep Neural Networkscited by 378
- 2018Insights on representational similarity in neural networks with canonical correlationcited by 318
- 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
- 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
4 more
- 2019RWTH ASR Systems for LibriSpeech: Hybrid vs Attention -- w/o Data Augmentationcited by 63
- 2021What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structurecited by 52
- 2019Espresso: A Fast End-to-end Neural Speech Recognition Toolkitcited by 31
- 2022Insights on Neural Representations for End-to-End Speech Recognitioncited by 4
Built on by
Citing papers per year: 2023: 1.
1 paper.