mlsanityData to 7 Oct 2026

Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Models

arXiv 2501.05767 ↗First posted 10 Jan 2025cs.CL cs.AI cs.CVcited by 22

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The recent advancement of Multimodal Large Language Models (MLLMs) has significantly improved their fine-grained perception of single images and general comprehension across multiple images. However, existing MLLMs still face challenges in achieving precise grounding in complex multi-image scenarios. To address this, we first explore a Chain-of-Thought (CoT) framework that integrates single-image grounding with multi-image comprehension. While partially effective, it remains unstable and struggles to capture abstract visual information due to its non-end-to-end nature. Therefore, we introduce Migician, the first multi-image grounding model capable of performing free-form and accurate grounding across multiple images. To support this, we present the MGrounding-630k dataset, which comprises data for several multi-image grounding tasks derived from existing datasets, along with newly generated free-form grounding instruction-following data. Furthermore, we propose MIG-Bench, a comprehensive benchmark specifically designed for evaluating multi-image grounding capabilities. Experimental results demonstrate that our model achieves significantly superior multi-image grounding capabilities, outperforming the best existing MLLMs by 24.94% and even surpassing much larger 70B models. Our code, model, dataset, and benchmark are fully open-sourced at https://migician-vg.github.io/.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 53 references that are in the map. First posted on arXiv 10 Jan 2025.

Builds on

45 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs16 Oct
  2. 2024Griffon v2: Advancing Multimodal Perception with High-Resolution Scaling and Visual-Language Co-Referring14 Mar
  3. 2024GroundingGPT:Language Enhanced Multi-modal Grounding Model11 Jan
  4. 2023GLaMM: Pixel Grounding Large Multimodal Model6 Nov
  5. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularity11 Oct
  6. 2023Kosmos-2: Grounding Multimodal Large Language Models to the World26 Jun

Other references · 39

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  7. 2024Qwen2 Technical Reportcited by 2,076
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  10. 2022Editing Models with Task Arithmeticcited by 1,076
  11. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  12. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
27 more
  1. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  2. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  3. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 571
  4. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 531
  5. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  6. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 471
  7. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  8. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 424
  9. 2024InternLM2 Technical Reportcited by 391
  10. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 342
  11. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  12. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 300
  13. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 285
  14. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 262
  15. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  16. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  17. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 178
  18. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
  19. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 136
  20. 2023ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalitiescited by 117
  21. 2018Learning to Describe Differences Between Pairs of Similar Imagescited by 114
  22. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 61
  23. 2023Position-Enhanced Visual Instruction Tuning for Multimodal Large Language Modelscited by 59
  24. 2017Imagination improves Multimodal Translationcited by 38
  25. 2024MIBench: Evaluating Multimodal Large Language Models over Multiple Imagescited by 36
  26. 2019Distilling Translations with Visual Awarenesscited by 30
  27. 2024Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Modelscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 11.

22 papers.

Sort:

By area:

  1. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  2. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  3. 2025Thinking with Visual Abstract: Enhancing Multimodal Reasoning via Visual Abstractioncited by 15
  4. 2025Unveiling the Potential of Vision-Language-Action Models with Open-Ended Multimodal Instructionscited by 12
  5. 2025PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoningcited by 11
  6. 2025Improving the Reasoning of Multi-Image Grounding in MLLMs via Reinforcement Learningcited by 11
  7. 2026Imagination Helps Visual Reasoning, But Not Yet in Latent Spacecited by 10
  8. 2025MedSG-Bench: A Benchmark for Medical Image Sequences Groundingcited by 8
  9. 2025VP-Bench: A Comprehensive Benchmark for Visual Prompting in Multimodal Large Language Modelscited by 4
  10. 2025Training Multi-Image Vision Agents via End2End Reinforcement Learningcited by 4
  11. 2025MIRG-RL: Multi-Image Reasoning and Grounding with Reinforcement Learningcited by 2
  12. 2025RefBench-PRO: Perceptual and Reasoning Oriented Benchmark for Referring Expression Comprehensioncited by 0
10 more
  1. 2026RGBX-R1: Visual Modality Chain-of-Thought Guided Reinforcement Learning for Multimodal Groundingcited by 0
  2. 2026GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentationcited by 0
  3. 2026CGC: Compositional Grounded Contrast for Fine-Grained Multi-Image Understandingcited by 0
  4. 2026Does Seeing More Mean Knowing More? Mono-Anchored Advantage Normalization for Multi-Source Visual Reasoningcited by 0
  5. 2026DifferAD-R1: A Difference-Guided IndustrialAnomaly Localization with Multimodal LargeLanguage Modelscited by 0
  6. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Modelscited by 0
  7. 2026DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cuescited by 0
  8. 2026Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Groundingcited by 0
  9. 2026MinCU: A Fine-Grained Benchmark for Grounded Minimal-Change Understanding in Image Pairscited by 0
  10. 2026Rho: A Foundation for Efficiently Adaptable VLA Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models
  2. 2025Towards Understanding Visual Grounding in Visual Language Models
  3. 2025ExpVG: Investigating the Design Space of Visual Grounding in Multimodal Large Language Model
  4. 2024Toward Visual Grounding: A Survey
  5. 2025PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs
  6. 2026Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding
  7. 2026IoU-PD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models
  8. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding