mlsanityData to 14 Oct 2026

Machine Translation Verbosity Control for Automatic Dubbing

arXiv 2110.03847 ↗First posted 8 Oct 2021cs.CL cs.SD eess.AScited by 18

Not on the map. Why?

Abstract (arXiv)

Automatic dubbing aims at seamlessly replacing the speech in a video document with synthetic speech in a different language. The task implies many challenges, one of which is generating translations that not only convey the original content, but also match the duration of the corresponding utterances. In this paper, we focus on the problem of controlling the verbosity of machine translation output, so that subsequent steps of our automatic dubbing pipeline can generate dubs of better quality. We propose new methods to control the verbosity of MT output and compare them against the state of the art with both intrinsic and extrinsic evaluations. For our experiments we use a public data set to dub English speeches into French, Italian, German and Spanish. Finally, we report extensive subjective tests that measure the impact of MT verbosity control on the final quality of dubbed video clips.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 19 of its 32 references that are in the map. First posted on arXiv 8 Oct 2021.

Builds on

19 of 32 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  5. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  6. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  7. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  8. 2017Controllable Abstractive Summarizationcited by 110
  9. 2020Speech Translation and the End-to-End Promise: Taking Stock of Where We Arecited by 77
  10. 2016Controlling Output Length in Neural Encoder-Decoderscited by 75
  11. 2019Positional Encoding to Control Output Sequence Lengthcited by 56
  12. 2019Controlling the Output Length of Neural Machine Translationcited by 36
7 more
  1. 2019Controlling Text Complexity in Neural Machine Translationcited by 32
  2. 2018Extreme Adaptation for Personalized Neural Machine Translationcited by 27
  3. 2020From Speech-to-Speech Translation to Automatic Dubbingcited by 25
  4. 2018Effect of data reduction on sequence-to-sequence neural TTScited by 16
  5. 2019Prosodic Phrase Alignment for Machine Dubbingcited by 13
  6. 2019In Other News: A Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Datacited by 7
  7. 2020Machine Translation with Unsupervised Length-Constraintscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 3; 2024: 3; 2025: 3; 2026: 1.

18 papers.

Sort:

By area:

  1. 2022CoCoA-MT: A Dataset and Benchmark for Contrastive Controlled MT with Application to Formalitycited by 24
  2. 2022VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbingcited by 20
  3. 2021Isometric MT: Neural Machine Translation for Automatic Dubbingcited by 18
  4. 2022Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbingcited by 17
  5. 2021Isochrony-Aware Neural Machine Translation for Automatic Dubbingcited by 14
  6. 2022Direct Speech Translation for Automatic Subtitlingcited by 7
  7. 2024ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Videocited by 6
  8. 2023Jointly Optimizing Translations and Speech Timing to Improve Isochrony in Automatic Dubbingcited by 5
  9. 2024Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learningcited by 4
  10. 2022Controlling Extra-Textual Attributes about Dialogue Participants -- A Case Study of English-to-Polish Neural Machine Translationcited by 3
  11. 2022AppTek's Submission to the IWSLT 2022 Isometric Spoken Language Translation Taskcited by 3
  12. 2023Improving Isochronous Machine Translation with Target Factors and Auxiliary Counterscited by 3
6 more
  1. 2025Prompting LLMs: Length Control for Isometric Machine Translationcited by 2
  2. 2024IsoChronoMeter: A simple and effective isochronic translation evaluation metriccited by 1
  3. 2025SparQLe: Speech Queries to Text Translation Through LLMscited by 1
  4. 2025Length Aware Speech Translation for Video Dubbingcited by 1
  5. 2023Evaluating the Efficacy of Length-Controllable Machine Translationcited by 0
  6. 2026PS-TTS: Phonetic Synchronization in Text-to-Speech for Achieving Natural Automated Dubbingcited by 0