mlsanityData to 14 Oct 2026

Timage: A Generative Text-in-Image Paradigm for Fine-Tuning Vision-Language Models

arXiv 2606.19944 ↗First posted 18 Jun 2026cs.CVcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal Large Language Models (MLLMs) often lose track of the right image regions during fine-grained spatial reasoning, because a textual query rarely carries any explicit geometric anchor into the pixel domain. Prevailing remedies either rewire the model's weights or pad the prompt with verbose instructions, yet neither reliably pins the language to the correct visual coordinates without eroding the backbone's general competence. We introduce Timage, a paradigm that recasts multimodal understanding as an alignment problem solved at the input: the query is drawn, as a typeset overlay, onto the image itself. The placement and appearance of this overlay are produced by a Constrained Schr"odinger Bridge (cSB), an entropic optimal-transport sampler that factorizes layout synthesis into two coupled stochastic stages. The first stage, Region Search, transports noise toward query-aligned image zones while obeying a hard occlusion barrier that protects salient foreground content; the second stage, Appearance Shaping, sizes the glyphs through an ``ink-budget'' regularizer so that the rendered text stays legible and visually balanced. The resulting overlay behaves as an explicit attention beacon that channels the model's focus along spatial semantics. On the VMCBench suite, Timage paired with a modest 7B backbone clearly overtakes far larger proprietary systems as well as parameter-tuned baselines. The study positions deliberate input reconstruction as a powerful, architecture-neutral lever for strengthening multimodal reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 57 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

48 of 57 references.

References · 48

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  11. 2022Visual Prompt Tuningcited by 1,681
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
36 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  2. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,026
  3. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  4. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  5. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
  6. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  7. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  8. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
  9. 2023LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 324
  10. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 320
  11. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 294
  12. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 230
  13. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translationcited by 213
  14. 2023LCM-LoRA: A Universal Stable-Diffusion Acceleration Modulecited by 211
  15. 2024InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learningcited by 198
  16. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 185
  17. 2024LoRA-GA: Low-Rank Adaptation with Gradient Approximationcited by 141
  18. 2023Parallel Sampling of Diffusion Modelscited by 112
  19. 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 104
  20. 2022Visual Prompt Tuning for Test-time Domain Adaptationcited by 97
  21. 2023Unpaired Image-to-Image Translation via Neural Schr"odinger Bridgecited by 96
  22. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 96
  23. 2022Visual Prompt Tuning for Generative Transfer Learningcited by 75
  24. 2017Parallel Multiscale Autoregressive Density Estimationcited by 71
  25. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  26. 2023Diversity-Aware Meta Visual Promptingcited by 58
  27. 2023StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generationcited by 58
  28. 2024Light and Optimal Schr"odinger Bridge Matchingcited by 45
  29. 2024MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learningcited by 44
  30. 2025Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluationcited by 30
  31. 2023Learning Expressive Prompting With Residuals for Vision Transformerscited by 21
  32. 2024Discrete Diffusion Schr"odinger Bridge Matching for Graph Transformationcited by 21
  33. 2025Audio Super-Resolution with Latent Bridge Modelscited by 14
  34. 2025Finding Local Diffusion Schr"odinger Bridge using Kolmogorov-Arnold Networkcited by 8
  35. 2025Cure or Poison? Embedding Instructions Visually Alters Hallucination in Vision-Language Modelscited by 4
  36. 2024Latent Schrodinger Bridge: Prompting Latent Diffusion for Fast Unpaired Image-to-Image Translationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models
  2. 2026SPAR: Semantic-Pixel Self-Alignment and Adaptive Routing for Unified Multimodal Models
  3. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models
  4. 2025Growing Visual Generative Capacity for Pre-Trained MLLMs
  5. 2025Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
  6. 2025Visual Representation Alignment for Multimodal Large Language Models
  7. 2026LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
  8. 2024The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models