# Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation

arXiv: https://arxiv.org/abs/2606.25432 · HTML: https://mlsanity.com/paper/2606.25432/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 17 of 17)

- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [A Long Way to Go: Investigating Length Correlations in RLHF](https://mlsanity.com/paper/2310.03716/) (2023)
- [Disentangling Length from Quality in Direct Preference Optimization](https://mlsanity.com/paper/2403.19159/) (2024)
- [MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale](https://mlsanity.com/paper/2412.05237/) (2024)
- [Correcting Length Bias in Neural Machine Translation](https://mlsanity.com/paper/1808.10006/) (2018)
- [Cost-of-Pass: An Economic Framework for Evaluating Language Models](https://mlsanity.com/paper/2504.13359/) (2025)
- [Post-hoc Reward Calibration: A Case Study on Length Bias](https://mlsanity.com/paper/2409.17407/) (2024)
- [Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models](https://mlsanity.com/paper/2502.01419/) (2025)
- [OckBench: Measuring the Efficiency of LLM Reasoning](https://mlsanity.com/paper/2511.05722/) (2025)
- [CogniLoad: A Synthetic Natural Language Reasoning Benchmark With Tunable Length, Intrinsic Difficulty, and Distractor Density](https://mlsanity.com/paper/2509.18458/) (2025)
- [The Price of Progress: Price Performance and the Future of AI](https://mlsanity.com/paper/2511.23455/) (2025)
- [Beyond Accuracy: Decomposing the Reasoning Efficiency of LLMs](https://mlsanity.com/paper/2602.09805/) (2026)
- [20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone](https://mlsanity.com/paper/2605.11405/) (2026)

## Abstract (arXiv)

Inference efficiency is typically pursued by shrinking the model: distillation, pruning, quantization, and sparse routing each lower per-token cost while treating token count as fixed. But output length has been inflating, and it is precisely the component the standard toolkit leaves untouched. Here, we argue that brevity is the missing inference-efficiency lever, and that pretraining data curation is a practical way to pull it: a model trained on concise, correct data learns to answer in fewer tokens; i.e. it has a lower Cost-of-Pass. We apply our VLM curation pipeline to the MAmmoTH-VL single-image subset, and compare models trained on our curated data, the standard MAmmoTH-VL data, and external open-weight frontier VLMs. On a controlled 20-evaluation set and 14 VLMs at 1B-4B activated parameters, we hold output length fixed with a per-model regression, separating brevity from quality, and price models in FLOPs per correct answer. Curation buys a 35x Cost-of-Pass advantage over the most verbose 4B comparator (Qwen3.5-4B) within ~1 pp of accuracy (0.41 vs 14.58 TFLOPs per correct answer; 0.691 vs 0.704 mean accuracy). Curation also buys a +17.55-percentage-point matched-length accuracy gain over the uncurated baseline that grows with model scale (from +16.7 pp at 1B to +21.2 pp at 4B). This brevity improvement concedes no quality: generic verbosity buys no accuracy at any capability or scale, and the window where reasoning-structured verbosity still earns its tokens shrinks from 4 of 8 capability groups at 2B to 1 of 8 at 4B. Per example, the concise model even reaches correct answers the verbose reasoning model misses, marking reasoning as a distinct curation target rather than something brevity gives up. Inference efficiency in this regime is a tokens-per-correct problem, and brevity is the lever that targets it directly.