mlsanityData to 7 Oct 2026

End-to-End Automatic Speech Translation of Audiobooks

arXiv 1802.04200 ↗First posted 12 Feb 2018cs.CLcited by 65

Not on the map. Why?

Abstract (arXiv)

We investigate end-to-end speech-to-text translation on a corpus of audiobooks specifically augmented for this task. Previous works investigated the extreme case where source language transcription is not available during learning nor decoding, but we also study a midway case where source language transcription is available at training time only. In this case, a single model is trained to decode source speech into target text in a single pass. Experimental results show that it is possible to train compact and efficient end-to-end speech translation models in this setup. We also distribute the corpus and hope that our speech translation baseline on this corpus will be challenged in the future.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 20; 2022: 18; 2023: 14; 2024: 6; 2025: 7.

65 papers.

Sort:

  1. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 118
  2. 2022Cross-modal Contrastive Learning for Speech Translationcited by 61
  3. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  4. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  5. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  6. 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
  7. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  8. 2021Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translationcited by 47
  9. 2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
  10. 2021CTC-based Compression for Direct Speech Translationcited by 45
  11. 2022Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentationcited by 43
  12. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
38 more
  1. 2023Transformers in Speech Processing: A Surveycited by 38
  2. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  3. 2022Large-Scale Streaming End-to-End Speech Translation with Neural Transducerscited by 31
  4. 2021Learning When to Translate for Streaming Speechcited by 30
  5. 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 27
  6. 2023Pre-training for Speech Translation: CTC Meets Optimal Transportcited by 25
  7. 2021Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translationcited by 23
  8. 2022WACO: Word-Aligned Contrastive Learning for Speech Translationcited by 21
  9. 2022Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translationcited by 20
  10. 2021End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021cited by 16
  11. 2021Speechformer: Reducing Information Loss in Direct Speech Translationcited by 15
  12. 2024Pushing the Limits of Zero-shot End-to-End Speech Translationcited by 14
  13. 2022Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence taskscited by 12
  14. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  15. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  16. 2022Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Datacited by 9
  17. 2023Tuning Large language model for End-to-end Speech Translationcited by 9
  18. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  19. 2022YFACC: A Yor`ub'a speech-image dataset for cross-lingual keyword localisation through visual groundingcited by 8
  20. 2023Efficient Monotonic Multihead Attentioncited by 8
  21. 2021Kosp2e: Korean Speech to English Translation Corpuscited by 7
  22. 2022Direct Speech Translation for Automatic Subtitlingcited by 7
  23. 2023Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learningcited by 7
  24. 2023Transfer Learning from Pre-trained Language Models Improves End-to-End Speech Summarizationcited by 7
  25. 2021Dealing with training and test segmentation mismatch: FBK@IWSLT2021cited by 6
  26. 2021Decision Attentive Regularization to Improve Simultaneous Speech Translation Systemscited by 6
  27. 2022Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translationcited by 6
  28. 2021Efficient Transformer for Direct Speech Translationcited by 5
  29. 2021A Unified Speaker Adaptation Approach for ASRcited by 5
  30. 2022On the Locality of Attention in Direct Speech Translationcited by 5
  31. 2022Does Joint Training Really Help Cascaded Speech Translation?cited by 5
  32. 2023A unified front-end framework for English text-to-speech synthesiscited by 4
  33. 2024CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleavingcited by 4
  34. 2025Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparisoncited by 4
  35. 2022AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translationcited by 3
  36. 2023Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translationcited by 3
  37. 2023Improving speech translation by fusing speech and textcited by 3
  38. 2023Enhancing End-to-End Conversational Speech Translation Through Target Language Context Utilizationcited by 3

50 most cited of 65 All 65 (JSON)