mlsanityData to 7 Oct 2026

Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognition

arXiv 2310.13015 ↗First posted 17 Oct 2023cs.CL cs.AI eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Adapters are an efficient, composable alternative to full fine-tuning of pre-trained models and help scale the deployment of large ASR models to many tasks. In practice, a task ID is commonly prepended to the input during inference to route to single-task adapters for the specified task. However, one major limitation of this approach is that the task ID may not be known during inference, rendering it unsuitable for most multi-task settings. To address this, we propose three novel task-ID-free methods to combine single-task adapters in multi-task ASR and investigate two learning algorithms for training. We evaluate our methods on 10 test sets from 4 diverse ASR tasks and show that our methods are non-destructive and parameter-efficient. While only updating 17% of the model parameters, our methods can achieve an 8% mean WER improvement relative to full fine-tuning and are on-par with task-ID adapter routing.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 17 Oct 2023.

Builds on

26 of 37 references.

References · 26

  1. 2016Layer Normalizationcited by 5,579
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  6. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  7. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  8. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  9. 2021A Survey of Transformerscited by 399
  10. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
  11. 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 214
  12. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
14 more
  1. 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
  2. 2020Hybrid Autoregressive Transducer (hat)cited by 106
  3. 2020FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularizationcited by 66
  4. 2020A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Taskscited by 51
  5. 2021Exploiting Adapters for Cross-lingual Low-resource Speech Recognitioncited by 33
  6. 2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 31
  7. 2017Transfer Learning for Speech Recognition on a Budgetcited by 25
  8. 2022Using Adapters to Overcome Catastrophic Forgetting in End-to-End Automatic Speech Recognitioncited by 15
  9. 2021E2E-based Multi-task Learning Approach to Joint Speech and Accent Recognitioncited by 12
  10. 2021Transformer-Based Models for Question Answering on COVID19cited by 7
  11. 2023Modular Domain Adaptation for Conformer-Based Streaming ASRcited by 7
  12. 2020An Experimental Evaluation of Transformer-based Language Models in the Biomedical Domaincited by 3
  13. 2022Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluationcited by 3
  14. 2023A Comparison of Semi-Supervised Learning Techniques for Streaming ASR at Scalecited by 1

Built on by

Not cited yet.