# Evaluating the Efficacy of Length-Controllable Machine Translation

arXiv: https://arxiv.org/abs/2305.02300 · HTML: https://mlsanity.com/paper/2305.02300/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 3 May 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text](https://mlsanity.com/paper/2202.06935/) (2022)
- [Controllable Abstractive Summarization](https://mlsanity.com/paper/1711.05217/) (2017)
- [Rethinking embedding coupling in pre-trained language models](https://mlsanity.com/paper/2010.12821/) (2020)
- [Controlling Output Length in Neural Encoder-Decoders](https://mlsanity.com/paper/1609.09552/) (2016)
- [A Statistical Analysis of Summarization Evaluation Metrics using Resampling Methods](https://mlsanity.com/paper/2104.00054/) (2021)
- [Controlling the Output Length of Neural Machine Translation](https://mlsanity.com/paper/1910.10408/) (2019)
- [Length-Controllable Image Captioning](https://mlsanity.com/paper/2007.09580/) (2020)
- [From Speech-to-Speech Translation to Automatic Dubbing](https://mlsanity.com/paper/2001.06785/) (2020)
- [FFCI: A Framework for Interpretable Automatic Evaluation of Summarization](https://mlsanity.com/paper/2011.13662/) (2020)
- [Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extraction](https://mlsanity.com/paper/2005.01791/) (2020)
- [Is 42 the Answer to Everything in Subtitling-oriented Speech Translation?](https://mlsanity.com/paper/2006.01080/) (2020)
- [Machine Translation Verbosity Control for Automatic Dubbing](https://mlsanity.com/paper/2110.03847/) (2021)
- [Isometric MT: Neural Machine Translation for Automatic Dubbing](https://mlsanity.com/paper/2112.08682/) (2021)
- [Evaluating the Efficacy of Summarization Evaluation across Languages](https://mlsanity.com/paper/2106.01478/) (2021)
- [Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarization](https://mlsanity.com/paper/2205.14521/) (2022)
- [Explicit Sentence Compression for Neural Machine Translation](https://mlsanity.com/paper/1912.11980/) (2019)
- [Length-controllable Abstractive Summarization by Guiding with Summary Prototype](https://mlsanity.com/paper/2001.07331/) (2020)
- [Alternative Weighting Schemes for ELMo Embeddings](https://mlsanity.com/paper/1904.02954/) (2019)
- [Machine Translation with Unsupervised Length-Constraints](https://mlsanity.com/paper/2004.03176/) (2020)

## Abstract (arXiv)

Length-controllable machine translation is a type of constrained translation. It aims to contain the original meaning as much as possible while controlling the length of the translation. We can use automatic summarization or machine translation evaluation metrics for length-controllable machine translation, but this is not necessarily suitable and accurate. This work is the first attempt to evaluate the automatic metrics for length-controllable machine translation tasks systematically. We conduct a rigorous human evaluation on two translation directions and evaluate 18 summarization or translation evaluation metrics. We find that BLEURT and COMET have the highest correlation with human evaluation and are most suitable as evaluation metrics for length-controllable machine translation.