mlsanityData to 7 Oct 2026

Curriculum Pre-training for End-to-End Speech Translation

arXiv 2004.10093 ↗First posted 21 Apr 2020cs.CL cs.SD eess.AScited by 57

Not on the map. Why?

Abstract (arXiv)

End-to-end speech translation poses a heavy burden on the encoder, because it has to transcribe, understand, and learn cross-lingual semantics simultaneously. To obtain a powerful encoder, traditional methods pre-train it on ASR data to capture speech features. However, we argue that pre-training the encoder only through simple speech recognition is not enough and high-level linguistic knowledge should be considered. Inspired by this, we propose a curriculum pre-training method that includes an elementary course for transcription learning and two advanced courses for understanding the utterance and mapping words in two languages. The difficulty of these courses is gradually increasing. Experiments show that our curriculum pre-training method leads to significant improvements on En-De and En-Fr speech translation benchmarks.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 12; 2023: 17; 2024: 8; 2025: 5; 2026: 2.

57 papers.

Sort:

By area:

  1. 2021Curriculum Learning: A Surveycited by 257
  2. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  3. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  4. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  5. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  6. 2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
  7. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
  8. 2021Learning When to Translate for Streaming Speechcited by 30
  9. 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 27
  10. 2024Denoising Task Difficulty-based Curriculum for Training Diffusion Modelscited by 26
  11. 2023Pre-training for Speech Translation: CTC Meets Optimal Transportcited by 25
  12. 2022CLNode: Curriculum Learning for Node Classificationcited by 24
38 more
  1. 2023CMOT: Cross-modal Mixup via Optimal Transport for Speech Translationcited by 22
  2. 2022WACO: Word-Aligned Contrastive Learning for Speech Translationcited by 21
  3. 2021Regularizing End-to-End Speech Translation with Triangular Decomposition Agreementcited by 16
  4. 2022Weakly-supervised Temporal Path Representation Learning with Contrastive Curriculum Learning -- Extended Versioncited by 15
  5. 2023Understanding and Bridging the Modality Gap for Speech Translationcited by 15
  6. 2023Recent Advances in Direct Speech-to-text Translationcited by 15
  7. 2022Non-Parametric Domain Adaptation for End-to-End Speech Translationcited by 12
  8. 2024LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Modelscited by 12
  9. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  10. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  11. 2022Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Datacited by 9
  12. 2023Back Translation for Speech-to-text Translation Without Transcriptscited by 9
  13. 2023ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translationcited by 9
  14. 2023CTC-based Non-autoregressive Speech Translationcited by 9
  15. 2023Tuning Large language model for End-to-end Speech Translationcited by 9
  16. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  17. 2023Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learningcited by 7
  18. 2023Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarizationcited by 7
  19. 2022On the Impact of Noises in Crowd-Sourced Data for Speech Translationcited by 6
  20. 2022Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translationcited by 6
  21. 2023Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inferencecited by 6
  22. 2023An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translationcited by 6
  23. 2023Rethinking and Improving Multi-task Learning for End-to-end Speech Translationcited by 6
  24. 2022DiscrimLoss: A Universal Loss for Hard Samples and Incorrect Samples Discriminationcited by 5
  25. 2022Generating Synthetic Speech from SpokenVocab for Speech Translationcited by 5
  26. 2024CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleavingcited by 4
  27. 2024Benchmarking Japanese Speech Recognition on ASR-LLM Setups with Multi-Pass Augmented Generative Error Correctioncited by 4
  28. 2021The Volctrans Neural Speech Translation System for IWSLT 2021cited by 3
  29. 2022AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translationcited by 3
  30. 2023Improving speech translation by fusing speech and textcited by 3
  31. 2021IWA: Integrated Gradient based White-box Attacks for Fooling Deep Neural Networkscited by 2
  32. 2021AlloST: Low-resource Speech Translation without Source Transcriptioncited by 2
  33. 2024Scheduled Curiosity-Deep Dyna-Q: Efficient Exploration for Dialog Policy Learningcited by 2
  34. 2025Speech Translation Refinement using Large Language Modelscited by 2
  35. 2026Demystifying Data Organization for Enhanced LLM Trainingcited by 2
  36. 2023Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translationcited by 1
  37. 2023Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNscited by 1
  38. 2024False Positive Sampling-based Data Augmentation for Enhanced 3D Object Detection Accuracycited by 1

50 most cited of 57 All 57 (JSON)