mlsanityData to 7 Oct 2026

On the Implications of Verbose LLM Outputs: A Case Study in Translation Evaluation

arXiv 2410.00863 ↗First posted 1 Oct 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

This paper investigates the impact of verbose LLM translations on evaluation. We first demonstrate the prevalence of this behavior across several LLM outputs drawn from the WMT 2024 general shared task on machine translation. We then identify the primary triggers of verbosity, including safety, copyright concerns, and insufficient context in short input queries. Finally, we show that ignoring this behavior unfairly penalizes more verbose LLMs according to both automatic and human evaluations, highlighting the need to address this issue for more accurate future evaluations.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 7 of its 14 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

7 of 14 references.

References · 7

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  3. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  4. 2024Chain-of-Thought Reasoning Without Promptingcited by 208
  5. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  6. 2024Preliminary WMT24 Ranking of General MT Systems and LLMscited by 20
  7. 2024Is Context Helpful for Chat Translation Evaluation?cited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 10; 2026: 2.

13 papers.

Sort:

By area:

  1. 2025MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talkscited by 14
  2. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translationcited by 8
  3. 2025TeachLM: Post-Training LLMs for Education Using Authentic Learning Datacited by 8
  4. 2024Verbosity neq Veracity: Demystify Verbosity Compensation Behavior of Large Language Modelscited by 5
  5. 2025Synthetic Fluency: Hallucinations, Confabulations, and the Creation of Irish Words in LLM-Generated Translationscited by 2
  6. 2025ConCISE: A Reference-Free Conciseness Evaluation Metric for LLM-Generated Answerscited by 1
  7. 2025THELMA: Task Based Holistic Evaluation of Large Language Model Applications-RAG Question Answeringcited by 0
  8. 2025GDC Cohort Copilot: An AI Copilot for Curating Cohorts from the Genomic Data Commonscited by 0
  9. 2025Humans learn to prefer trustworthy AI over human partnerscited by 0
  10. 2025Sanitize Your Responses: Mitigating Privacy Leakage in Large Language Modelscited by 0
  11. 2025When the Gold Standard Isn't Necessarily Standard: Challenges of Evaluating the Translation of User-Generated Contentcited by 0
  12. 2026HOMURA: Taming the Sand-Glass for Time-Constrained LLM Translation via Reinforcement Learningcited by 0
1 more
  1. 2026AnalyticsGPT: An LLM Workflow for Scientometric Question Answeringcited by 0