# On the Implications of Verbose LLM Outputs: A Case Study in Translation Evaluation

arXiv: https://arxiv.org/abs/2410.00863 · HTML: https://mlsanity.com/paper/2410.00863/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 1 Oct 2024.

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [Chain-of-Thought Reasoning Without Prompting](https://mlsanity.com/paper/2402.10200/) (2024)
- [Aya 23: Open Weight Releases to Further Multilingual Progress](https://mlsanity.com/paper/2405.15032/) (2024)
- [Preliminary WMT24 Ranking of General MT Systems and LLMs](https://mlsanity.com/paper/2407.19884/) (2024)
- [Is Context Helpful for Chat Translation Evaluation?](https://mlsanity.com/paper/2403.08314/) (2024)

## Built on by (most cited first, 13 of 13)

- [MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks](https://mlsanity.com/paper/2507.19634/) (2025)
- [TeachLM: Post-Training LLMs for Education Using Authentic Learning Data](https://mlsanity.com/paper/2510.05087/) (2025)
- [Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation](https://mlsanity.com/paper/2504.01919/) (2025)
- [Verbosity neq Veracity: Demystify Verbosity Compensation Behavior of Large Language Models](https://mlsanity.com/paper/2411.07858/) (2024)
- [Synthetic Fluency: Hallucinations, Confabulations, and the Creation of Irish Words in LLM-Generated Translations](https://mlsanity.com/paper/2504.07680/) (2025)
- [Sanitize Your Responses: Mitigating Privacy Leakage in Large Language Models](https://mlsanity.com/paper/2509.24488/) (2025)
- [ConCISE: A Reference-Free Conciseness Evaluation Metric for LLM-Generated Answers](https://mlsanity.com/paper/2511.16846/) (2025)
- [HOMURA: Taming the Sand-Glass for Time-Constrained LLM Translation via Reinforcement Learning](https://mlsanity.com/paper/2601.10187/) (2026)
- [THELMA: Task Based Holistic Evaluation of Large Language Model Applications-RAG Question Answering](https://mlsanity.com/paper/2505.11626/) (2025)
- [GDC Cohort Copilot: An AI Copilot for Curating Cohorts from the Genomic Data Commons](https://mlsanity.com/paper/2507.02221/) (2025)
- [Humans learn to prefer trustworthy AI over human partners](https://mlsanity.com/paper/2507.13524/) (2025)
- [When the Gold Standard Isn't Necessarily Standard: Challenges of Evaluating the Translation of User-Generated Content](https://mlsanity.com/paper/2512.17738/) (2025)
- [AnalyticsGPT: An LLM Workflow for Scientometric Question Answering](https://mlsanity.com/paper/2602.09817/) (2026)

## Abstract (arXiv)

This paper investigates the impact of verbose LLM translations on evaluation. We first demonstrate the prevalence of this behavior across several LLM outputs drawn from the WMT 2024 general shared task on machine translation. We then identify the primary triggers of verbosity, including safety, copyright concerns, and insufficient context in short input queries. Finally, we show that ignoring this behavior unfairly penalizes more verbose LLMs according to both automatic and human evaluations, highlighting the need to address this issue for more accurate future evaluations.