mlsanityData to 14 Oct 2026

Dense Coordinate-List Fine-Tuning Induces a Controllable Interference Surface in Vision-Language Models

arXiv 2606.14507 ↗First posted 12 Jun 2026cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Fine-tuning vision-language models to emit dense coordinate lists improves visual grounding but also changes how models serialize, repeat, and terminate structured outputs. We study this behavior as a generation and control surface. In Gemma 4 12B, high-capacity q/k/v/o LoRA raises class-aware [email protected] from 0.007 to 0.448 while inducing repeated-tail pressure (duplicate rate 0.080, max repeat 23). A q/v rank sweep keeps max repeat at 21-22 across ranks 4-64, showing capacity persistence. The target signal is separable: object-level repeat-stop removes exact repeated records (duplicate rate 0.000, max repeat 1) while preserving F1 (0.494 to 0.490) and stricter [email protected] (0.381 to 0.385). Structure-axis probes localize the effect to bbox-coordinate object lists; dense non-bbox and spatial/count JSON remain repeat-clean, including under high-capacity adapters. Qwen3-VL-8B reproduces a clean controlled endpoint ([email protected] 0.318, duplicate rate 0.000), and COCO 2017 reproduces acquisition plus duplicate pressure. Dense coordinate-list adaptation therefore creates a structure-bound, cross-family interference surface that can be measured and controlled.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 15 of its 19 references that are in the map. First posted on arXiv 12 Jun 2026.

Builds on

15 of 19 references.

References · 15

  1. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  3. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  4. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  5. 2018Object Hallucination in Image Captioningcited by 669
  6. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  7. 2022Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generationcited by 94
  8. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 83
  9. 2024Vision-Flan: Scaling Human-Labeled Tasks in Visual Instruction Tuningcited by 73
  10. 2023Continual Instruction Tuning for Large Multimodal Modelscited by 49
  11. 2024Multimodal Instruction Tuning with Conditional Mixture of LoRAcited by 36
  12. 2024An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Modelscited by 22
3 more
  1. 2024SMoLoRA: Exploring and Defying Dual Catastrophic Forgetting in Continual Visual Instruction Tuningcited by 19
  2. 2023InsPLAD: A Dataset and Benchmark for Power Line Asset Inspection in UAV Imagescited by 15
  3. 2025ExpVG: Investigating the Design Space of Visual Grounding in Multimodal Large Language Modelcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Not All Error Yields to Scale: Where Scaling Stops in Vision-Language Inference
  2. 2023Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability
  3. 2025Mitigating Coordinate Prediction Bias from Positional Encoding Failures
  4. 2025Rethinking Fine-Tuning: Unlocking Hidden Capabilities in Vision-Language Models
  5. 2025Synthetic Stimuli, Real Gains: Rethinking VLM Fine-Tuning Through Fully Controlled Data Generation
  6. 2026Seeing Beyond Redundancy: Task Complexity's Role in Vision Token Specialization in VLLMs
  7. 2025Adaptive Residual-Update Steering for Low-Overhead Hallucination Mitigation in Large Vision Language Models
  8. 2024Rethinking Overlooked Aspects in Vision-Language Models