mlsanityData to 7 Oct 2026

Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translation

arXiv 2406.10276 ↗First posted 12 Jun 2024cs.CL cs.SD eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Language-agnostic many-to-one end-to-end speech translation models can convert audio signals from different source languages into text in a target language. These models do not need source language identification, which improves user experience. In some cases, the input language can be given or estimated. Our goal is to use this additional language information while preserving the quality of the other languages. We accomplish this by introducing a simple and effective linear input network. The linear input network is initialized as an identity matrix, which ensures that the model can perform as well as, or better than, the original model. Experimental results show that the proposed method can successfully enhance the specified language, while keeping the language-agnostic ability of the many-to-one ST models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

20 of 37 references.

References · 20

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  3. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  4. 2016Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translationcited by 113
  5. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  6. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  7. 2020Speech Translation and the End-to-End Promise: Taking Stock of Where We Arecited by 77
  8. 2018End-to-End Automatic Speech Translation of Audiobookscited by 65
  9. 2020Developing RNN-T Models Surpassing High-Performance Hybrid Models with Customization Capabilitycited by 63
  10. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  11. 2016Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptationcited by 38
  12. 2022Large-Scale Streaming End-to-End Speech Translation with Neural Transducerscited by 31
8 more
  1. 2019Bridging the Gap Between Monaural Speech Enhancement and Recognition with Distortion-Independent Acoustic Modelingcited by 21
  2. 2023Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Taskscited by 20
  3. 2020Multitask Training with Text Data for End-to-End Speech Recognitioncited by 19
  4. 2022A Weakly-Supervised Streaming Multilingual Speech Model with Truly Zero-Shot Capabilitycited by 12
  5. 2023Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignmentscited by 6
  6. 2023DiariST: Streaming Speech Translation with Speaker Diarizationcited by 5
  7. 2023Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translationcited by 4
  8. 2020Speaker Separation Using Speaker Inventories and Estimated Speechcited by 3

Built on by

Not cited yet.