mlsanityData to 14 Oct 2026

Perception in Reflection

arXiv 2504.07165 ↗First posted 9 Apr 2025cs.CVcited by 13

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We present a perception in reflection paradigm designed to transcend the limitations of current large vision-language models (LVLMs), which are expected yet often fail to achieve perfect perception initially. Specifically, we propose Reflective Perception (RePer), a dual-model reflection mechanism that systematically alternates between policy and critic models, enables iterative refinement of visual perception. This framework is powered by Reflective Perceptual Learning (RPL), which reinforces intrinsic reflective capabilities through a methodically constructed visual reflection dataset and reflective unlikelihood training. Comprehensive experimental evaluation demonstrates RePer's quantifiable improvements in image understanding, captioning precision, and hallucination reduction. Notably, RePer achieves strong alignment between model attention patterns and human visual focus, while RPL optimizes fine-grained and free-form preference alignment. These advancements establish perception in reflection as a robust paradigm for future multimodal agents, particularly in tasks requiring complex reasoning and multi-step manipulation.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 62 of its 75 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

62 of 75 references.

References · 62

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 2017Mask R-CNNcited by 7,277
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  12. 2020Scaling Laws for Neural Language Modelscited by 6,586
38 more
  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  6. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  7. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  8. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  9. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  10. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  11. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  12. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  13. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  14. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  15. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  16. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  17. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  18. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  19. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  20. 2018nocaps: novel object captioning at scalecited by 444
  21. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  22. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  23. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  24. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  25. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 165
  26. 2023SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoningcited by 159
  27. 2024LLaVA-Critic: Learning to Evaluate Multimodal Modelscited by 134
  28. 2024DreamBench++: A Human-Aligned Benchmark for Personalized Image Generationcited by 131
  29. 2024Understanding the performance gap between online and offline alignment algorithmscited by 110
  30. 2024Rule Based Rewards for Language Model Safetycited by 108
  31. 2024Large Multimodal Agents: A Surveycited by 105
  32. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revisioncited by 98
  33. 2024LiPO: Listwise Preference Optimization through Learning-to-Rankcited by 96
  34. 2024Benchmarking and Improving Detail Image Captioncited by 81
  35. 2023ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuningcited by 65
  36. 2024OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Textcited by 59
  37. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  38. 2023Merlin:Empowering Multimodal LLMs with Foresight Mindscited by 46

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 410
  2. 2025Latent Visual Reasoningcited by 104
  3. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 91
  4. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 77
  5. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
  6. 2025Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoningcited by 34
  7. 2026STEP3-VL-10B Technical Reportcited by 32
  8. 2025AlphaOne: Reasoning Models Thinking Slow and Fast at Test Timecited by 29
  9. 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agentscited by 21
  10. 2025DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLMcited by 6
  11. 2026Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMscited by 1
  12. 2026PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perceptioncited by 1
1 more
  1. 2026VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Contextcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Bridging Modality Disconnect in Self-Reflection via Closed-Loop Visually Grounded Verification
  2. 2025Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Information
  3. 2026Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning
  4. 2024Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models
  5. 2025Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models
  6. 2026From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models
  7. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
  8. 2025Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs