# Dense Coordinate-List Fine-Tuning Induces a Controllable Interference Surface in Vision-Language Models

arXiv: https://arxiv.org/abs/2606.14507 · HTML: https://mlsanity.com/paper/2606.14507/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 15 of its 19 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 15 of 15)

- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation](https://mlsanity.com/paper/2206.02369/) (2022)
- [Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding](https://mlsanity.com/paper/2503.06287/) (2025)
- [Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuning](https://mlsanity.com/paper/2402.11690/) (2024)
- [Continual Instruction Tuning for Large Multimodal Models](https://mlsanity.com/paper/2311.16206/) (2023)
- [Multimodal Instruction Tuning with Conditional Mixture of LoRA](https://mlsanity.com/paper/2402.15896/) (2024)
- [An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models](https://mlsanity.com/paper/2406.05130/) (2024)
- [SMoLoRA: Exploring and Defying Dual Catastrophic Forgetting in Continual Visual Instruction Tuning](https://mlsanity.com/paper/2411.13949/) (2024)
- [InsPLAD: A Dataset and Benchmark for Power Line Asset Inspection in UAV Images](https://mlsanity.com/paper/2311.01619/) (2023)
- [ExpVG: Investigating the Design Space of Visual Grounding in Multimodal Large Language Model](https://mlsanity.com/paper/2508.08066/) (2025)

## Abstract (arXiv)

Fine-tuning vision-language models to emit dense coordinate lists improves visual grounding but also changes how models serialize, repeat, and terminate structured outputs. We study this behavior as a generation and control surface. In Gemma 4 12B, high-capacity q/k/v/o LoRA raises class-aware F1@0.3 from 0.007 to 0.448 while inducing repeated-tail pressure (duplicate rate 0.080, max repeat 23). A q/v rank sweep keeps max repeat at 21-22 across ranks 4-64, showing capacity persistence. The target signal is separable: object-level repeat-stop removes exact repeated records (duplicate rate 0.000, max repeat 1) while preserving F1 (0.494 to 0.490) and stricter F1@0.5 (0.381 to 0.385). Structure-axis probes localize the effect to bbox-coordinate object lists; dense non-bbox and spatial/count JSON remain repeat-clean, including under high-capacity adapters. Qwen3-VL-8B reproduces a clean controlled endpoint (F1@0.3 0.318, duplicate rate 0.000), and COCO 2017 reproduces acquisition plus duplicate pressure. Dense coordinate-list adaptation therefore creates a structure-bound, cross-family interference surface that can be measured and controlled.