mlsanityData to 7 Oct 2026

Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition

arXiv 2602.04486 ↗First posted 4 Feb 2026cs.CLcited by 6

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Grounded Multimodal Named Entity Recognition (GMNER) aims to extract text-based entities, assign them semantic categories, and ground them to corresponding visual regions. In this work, we explore the potential of Multimodal Large Language Models (MLLMs) to perform GMNER in an end-to-end manner, moving beyond their typical role as auxiliary tools within cascaded pipelines. Crucially, our investigation reveals a fundamental challenge: MLLMs exhibit modality bias, including visual bias and textual bias, which stems from their tendency to take unimodal shortcuts rather than rigorous cross-modal verification. To address this, we propose Modality-aware Consistency Reasoning (MCR), which enforces structured cross-modal reasoning through Multi-style Reasoning Schema Injection (MRSI) and Constraint-guided Verifiable Optimization (CVO). MRSI transforms abstract constraints into executable reasoning chains, while CVO empowers the model to dynamically align its reasoning trajectories with Group Relative Policy Optimization (GRPO). Experiments on GMNER and visual grounding tasks demonstrate that MCR effectively mitigates modality bias and achieves superior performance compared to existing baselines.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 57 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

28 of 57 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025MiMo-VL Technical Report4 Jun
  2. 2025ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL30 May
  3. 2025R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO22 May
  4. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  5. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thought13 Jan
  6. 2023Multimodal Chain-of-Thought Reasoning in Language Models2 Feb

Other references · 22

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  5. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  6. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 235
  7. 2024mDPO: Conditional Preference Optimization for Multimodal Large Language Modelscited by 92
  8. 2024The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audiocited by 48
  9. 2018Multimodal Named Entity Recognition for Short Social Media Postscited by 43
  10. 2023GREC: Generalized Referring Expression Comprehensioncited by 39
  11. 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
  12. 2025Reinforce-Ada: An Adaptive Sampling Framework under Non-linear RL Objectivescited by 23
10 more
  1. 2025Thinking in Character: Advancing Role-Playing Agents with Role-Aware Reasoningcited by 14
  2. 2025ExPO: Unlocking Hard Reasoning with Self-Explanation-Guided Reinforcement Learningcited by 13
  3. 2025Memory-augmented Query Reconstruction for LLM-based Knowledge Graph Reasoningcited by 10
  4. 2025ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Searchcited by 9
  5. 2024LLM with Relation Classifier for Document-Level Relation Extractioncited by 8
  6. 2024LLMs as Bridges: Reformulating Grounded Multimodal Named Entity Recognitioncited by 7
  7. 2025SAMChat: Introducing Chain of Thought Reasoning and GRPO to a Multimodal Small Language Model for Small Scale Remote Sensingcited by 6
  8. 2025LoCoT2V-Bench: Benchmarking Long-Form and Complex Text-to-Video Generationcited by 6
  9. 2024SCANNER: Knowledge-Enhanced Approach for Robust Multi-modal Named Entity Recognition of Unseen Entitiescited by 5
  10. 2025ReFineG: Synergizing Small Supervised Models and LLMs for Low-Resource Grounded Multimodal NERcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Proposal-Free Query-Guided Network for Grounded Multimodal Named Entity Recognition
  2. 2026E2E-GMNER: End-to-End Generative Grounded Multimodal Named Entity Recognition
  3. 2023Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction
  4. 2026SAKE: Self-aware Knowledge Exploitation-Exploration for Grounded Multimodal Named Entity Recognition
  5. 2024MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing
  6. 2026Mind the Gap No More: Achieving Zero-Gap Multimodal Integration via One Tokenizer
  7. 2025Some Modalities are More Equal Than Others: Decoding and Architecting Multimodal Integration in MLLMs
  8. 2025Exploring and Evaluating Multimodal Knowledge Reasoning Consistency of Multimodal Large Language Models