mlsanityData to 14 Oct 2026

Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adapter

arXiv 2309.09443 ↗First posted 18 Sep 2023eess.AS cs.CL cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Multilingual intelligent assistants, such as ChatGPT, have recently gained popularity. To further expand the applications of multilingual artificial intelligence assistants and facilitate international communication, it is essential to enhance the performance of multilingual speech recognition, which is a crucial component of speech interaction. In this paper, we propose two simple and parameter-efficient methods: language prompt tuning and frame-level language adapter, to respectively enhance language-configurable and language-agnostic multilingual speech recognition. Additionally, we explore the feasibility of integrating these two approaches using parameter-efficient fine-tuning methods. Our experiments demonstrate significant performance improvements across seven languages using our proposed methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 256 of its 412 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

256 of 412 references.

References · 256

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  10. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  11. 2017Graph Attention Networkscited by 8,923
  12. 2023Segment Anythingcited by 8,234
38 more
  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2017Mask R-CNNcited by 7,277
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  8. 2016Layer Normalizationcited by 5,669
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  11. 2017Squeeze-and-Excitation Networkscited by 4,789
  12. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  13. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  14. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  15. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  16. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  17. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  18. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  19. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  20. 2018YOLOv3: An Incremental Improvementcited by 2,659
  21. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  22. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  23. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  24. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  25. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  26. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  27. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  28. 2021ViViT: A Video Vision Transformercited by 1,620
  29. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  30. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  31. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  32. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  33. 2020Point Transformercited by 1,470
  34. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  35. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  36. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  37. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  38. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369

50 most cited of 256 All 256 (JSON)

Built on by

Not cited yet.