mlsanityData to 7 Oct 2026

Tied Multitask Learning for Neural Speech Translation

arXiv 1802.06655 ↗First posted 19 Feb 2018cs.CLcited by 54

Not on the map. Why?

Abstract (arXiv)

We explore multitask models for neural translation of speech, augmenting them in order to reflect two intuitive notions. First, we introduce a model where the second task decoder receives information from the decoder of the first task, since higher-level intermediate representations should provide useful information. Second, we apply regularization that encourages transitivity and invertibility. We show that the application of these notions on jointly trained models improves performance on the tasks of low-resource speech transcription and translation. It also leads to better performance when using attention information for word discovery over unsegmented input.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 15; 2023: 14; 2024: 4; 2025: 3; 2026: 1.

54 papers.

Sort:

By area:

  1. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  2. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  3. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  4. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  5. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  6. 2022Information-Transport-based Policy for Simultaneous Translationcited by 40
  7. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
  8. 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 27
  9. 2022End-to-End Speech Translation for Code Switched Speechcited by 26
  10. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
  11. 2023Pre-training for Speech Translation: CTC Meets Optimal Transportcited by 25
  12. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
38 more
  1. 2021Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translationcited by 23
  2. 2023End-to-End Simultaneous Speech Translation with Differentiable Segmentationcited by 22
  3. 2022Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translationcited by 20
  4. 2021Regularizing End-to-End Speech Translation with Triangular Decomposition Agreementcited by 16
  5. 2023Recent Advances in Direct Speech-to-text Translationcited by 15
  6. 2024Pushing the Limits of Zero-shot End-to-End Speech Translationcited by 14
  7. 2022Non-Parametric Domain Adaptation for End-to-End Speech Translationcited by 12
  8. 2024LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Modelscited by 12
  9. 2021How to Split: the Effect of Word Segmentation on Gender Bias in Speech Translationcited by 11
  10. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  11. 2022Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Datacited by 9
  12. 2023CTC-based Non-autoregressive Speech Translationcited by 9
  13. 2023UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructionscited by 9
  14. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  15. 2022Who Are We Talking About? Handling Person Names in Speech Translationcited by 8
  16. 2021Between Flexibility and Consistency: Joint Generation of Captions and Subtitlescited by 7
  17. 2023Neural Machine Translation for the Indigenous Languages of the Americas: An Introductioncited by 7
  18. 2021Dealing with training and test segmentation mismatch: FBK@IWSLT2021cited by 6
  19. 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023cited by 6
  20. 2023Unified Segment-to-Segment Framework for Simultaneous Sequence Generationcited by 6
  21. 2023Rethinking and Improving Multi-task Learning for End-to-end Speech Translationcited by 6
  22. 2022Generating Synthetic Speech from SpokenVocab for Speech Translationcited by 5
  23. 2022Does Joint Training Really Help Cascaded Speech Translation?cited by 5
  24. 2023Constructing Time-Series Momentum Portfolios with Deep Multi-Task Learningcited by 5
  25. 2024A Case Study on Context-Aware Neural Machine Translation with Multi-Task Learningcited by 4
  26. 2025Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Reviewcited by 4
  27. 2022Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Datacited by 3
  28. 2021AlloST: Low-resource Speech Translation without Source Transcriptioncited by 2
  29. 2021One Source, Two Targets: Challenges and Rewards of Dual Decodingcited by 2
  30. 2021DeepAnalyze: Learning to Localize Crashes at Scalecited by 2
  31. 2022On the Effectiveness of Pinyin-Character Dual-Decoding for End-to-End Mandarin Chinese ASRcited by 2
  32. 2022Learning to Jointly Transcribe and Subtitle for End-to-End Spontaneous Speech Recognitioncited by 2
  33. 2022Efficient Speech Translation with Pre-trained Modelscited by 1
  34. 2023Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translationcited by 1
  35. 2023Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcriptscited by 1
  36. 2023A Case Study on Context Encoding in Multi-Encoder based Document-Level Neural Machine Translationcited by 1
  37. 2024A Modular-based Strategy for Mitigating Gradient Conflicts in Simultaneous Speech Translationcited by 1
  38. 2021The IWSLT 2021 BUT Speech Translation Systemscited by 0

50 most cited of 54 All 54 (JSON)