mlsanityData to 14 Oct 2026

LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation

arXiv 2604.11789 ↗First posted 13 Apr 2026cs.CVcited by 6

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Large Multimodal Models (LMMs) have achieved remarkable progress in general-purpose vision--language understanding, yet they remain limited in tasks requiring precise object-level grounding, fine-grained spatial reasoning, and controllable visual manipulation. In particular, existing systems often struggle to identify the correct instance, preserve object identity across interactions, and localize or modify designated regions with high precision. Object-centric vision provides a principled framework for addressing these challenges by promoting explicit representations and operations over visual entities, thereby extending multimodal systems from global scene understanding to object-level understanding, segmentation, editing, and generation. This paper presents a comprehensive review of recent advances at the convergence of LMMs and object-centric vision. We organize the literature into four major themes: object-centric visual understanding, object-centric referring segmentation, object-centric visual editing, and object-centric visual generation. We further summarize the key modeling paradigms, learning strategies, and evaluation protocols that support these capabilities. Finally, we discuss open challenges and future directions, including robust instance permanence, fine-grained spatial control, consistent multi-step interaction, unified cross-task modeling, and reliable benchmarking under distribution shift. We hope this paper provides a structured perspective on the development of scalable, precise, and trustworthy object-centric multimodal systems.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 234 of its 246 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

234 of 246 references.

Closest ancestors · 24

Same direction, newest first

  1. 2026Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning7 Apr
  2. 2026MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods29 Jan
  3. 2025Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation22 Oct
  4. 2025CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation13 Oct
  5. 2025Video models are zero-shot learners and reasoners24 Sep
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency25 Aug
  7. 2025Decoupled Seg Tokens Make Stronger Reasoning Video Segmenter and Grounder28 Jun
  8. 2025Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning27 Jun
  9. 2025RSVP: Reasoning Segmentation via Visual Prompting and Multi-modal Chain-of-Thought4 Jun
  10. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoning29 May
  11. 2025VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning17 May
  12. 2025GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentation10 Apr
12 more
  1. 2025POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation1 Apr
  2. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  3. 2025Pixel-Level Reasoning Segmentation via Multi-turn Conversations13 Feb
  4. 2025The Devil is in Temporal Token: High Quality Video Reasoning Segmentation15 Jan
  5. 2024PRIMA: Multi-Image Vision-Language Models for Reasoning Segmentation19 Dec
  6. 2024HyperSeg: Towards Universal Visual Segmentation with Large Language Model26 Nov
  7. 2024SegLLM: Multi-round Reasoning Segmentation24 Oct
  8. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos29 Sep
  9. 2024VISA: Reasoning Video Object Segmentation via Large Language Models16 Jul
  10. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding27 Jun
  11. 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
  12. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug

Other references · 210

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2017Neural Discrete Representation Learningcited by 4,712
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  12. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  2. 2023Qwen Technical Reportcited by 3,344
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  8. 2025Qwen3-VL Technical Reportcited by 2,793
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  12. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  13. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  14. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  15. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  16. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  17. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  18. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  19. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  20. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  21. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  22. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  23. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  24. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  25. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  26. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  27. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  28. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  29. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  30. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  31. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  32. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  33. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  34. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  35. 2025SAM 3: Segment Anything with Conceptscited by 906
  36. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  37. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  38. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843

50 most cited of 210 All 210 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Object-Uni: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation
  2. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models
  3. 2025PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding
  4. 2025Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models
  5. 2024Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Models
  6. 2026MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment
  7. 2026Perception-Aware Multimodal Spatial Reasoning from Monocular Images
  8. 2024Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models