mlsanityData to 7 Oct 2026

Bridging the Gap between Pre-Training and Fine-Tuning for End-to-End Speech Translation

arXiv 1909.07575 ↗First posted 17 Sep 2019cs.CL eess.AScited by 39

Not on the map. Why?

Abstract (arXiv)

End-to-end speech translation, a hot topic in recent years, aims to translate a segment of audio into a specific language with an end-to-end model. Conventional approaches employ multi-task learning and pre-training methods for this task, but they suffer from the huge gap between pre-training and fine-tuning. To address these issues, we propose a Tandem Connectionist Encoding Network (TCEN) which bridges the gap by reusing all subnets in fine-tuning, keeping the roles of subnets consistent, and pre-training the attention module. Furthermore, we propose two simple but effective methods to guarantee the speech encoder outputs and the MT encoder inputs are consistent in terms of semantic representation and sequence length. Experimental results show that our model outperforms baselines 2.2 BLEU on a large benchmark dataset.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 8; 2023: 12; 2024: 2; 2025: 2; 2026: 3.

39 papers.

Sort:

  1. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  2. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  3. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  4. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  5. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
  6. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  7. 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 27
  8. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
  9. 2023Pre-training for Speech Translation: CTC Meets Optimal Transportcited by 25
  10. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  11. 2023CMOT: Cross-modal Mixup via Optimal Transport for Speech Translationcited by 22
  12. 2021Regularizing End-to-End Speech Translation with Triangular Decomposition Agreementcited by 16
27 more
  1. 2022M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translationcited by 15
  2. 2023Understanding and Bridging the Modality Gap for Speech Translationcited by 15
  3. 2023HANSEN: Human and AI Spoken Text Benchmark for Authorship Analysiscited by 10
  4. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  5. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  6. 2022Adaptive multilingual speech recognition with pretrained modelscited by 9
  7. 2022Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Datacited by 9
  8. 2023Back Translation for Speech-to-text Translation Without Transcriptscited by 9
  9. 2023ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translationcited by 9
  10. 2023CTC-based Non-autoregressive Speech Translationcited by 9
  11. 2021An Approach to Improve Robustness of NLP Systems against ASR Errorscited by 8
  12. 2022SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Predictioncited by 8
  13. 2023Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarizationcited by 7
  14. 2022Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translationcited by 6
  15. 2023Modality Adaption or Regularization? A Case Study on End-to-End Speech Translationcited by 6
  16. 2023Rethinking and Improving Multi-task Learning for End-to-end Speech Translationcited by 6
  17. 2022Does Joint Training Really Help Cascaded Speech Translation?cited by 5
  18. 2023Improving speech translation by fusing speech and textcited by 3
  19. 2023Towards a Deep Understanding of Multilingual End-to-End Speech Translationcited by 3
  20. 2024Representation Purification for End-to-End Speech Translationcited by 3
  21. 2021AlloST: Low-resource Speech Translation without Source Transcriptioncited by 2
  22. 2025Adaptive Inner Speech-Text Alignment for LLM-based Speech Translationcited by 2
  23. 2024A Modular-based Strategy for Mitigating Gradient Conflicts in Simultaneous Speech Translationcited by 1
  24. 2025AdaST: Dynamically Adapting Encoder States in the Decoder for End-to-End Speech-to-Text Translationcited by 0
  25. 2026Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Textcited by 0
  26. 2026FACT: A Simple and Efficient Framework for Active Finetuningcited by 0
  27. 2026Align, Integrate, and Fire: Efficient Token-Level Alignment for Zero-Shot SpeechLLMscited by 0