mlsanityData to 7 Oct 2026

Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-off

arXiv 2602.23730 ↗First posted 27 Feb 2026cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Recent advancements in Multimodal Large Language Models (MLLMs) pursue omni-perception capabilities, yet integrating robust sensory grounding with complex reasoning remains a challenge, particularly for underrepresented regions. In this report, we introduce the research preview of MERaLiON2-Omni (Alpha), a 10B-parameter multilingual omni-perception tailored for Southeast Asia (SEA). We present a progressive training pipeline that explicitly decouples and then integrates "System 1" (Perception) and "System 2" (Reasoning) capabilities. First, we establish a robust Perception Backbone by aligning region-specific audio-visual cues (e.g., Singlish code-switching, local cultural landmarks) with a multilingual LLM through orthogonal modality adaptation. Second, to inject cognitive capabilities without large-scale supervision, we propose a cost-effective Generate-Judge-Refine pipeline. By utilizing a Super-LLM to filter hallucinations and resolve conflicts via a consensus mechanism, we synthesize high-quality silver data that transfers textual Chain-of-Thought reasoning to multimodal scenarios. Comprehensive evaluation on our newly introduced SEA-Omni Benchmark Suite reveals an Efficiency-Stability Paradox: while reasoning acts as a non-linear amplifier for abstract tasks (boosting mathematical and instruction-following performance significantly), it introduces instability in low-level sensory processing. Specifically, we identify Temporal Drift in long-context audio, where extended reasoning desynchronizes the model from acoustic timestamps, and Visual Over-interpretation, where logic overrides pixel-level reality. This report details the architecture, the data-efficient training recipe, and a diagnostic analysis of the trade-offs between robust perception and structured reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 34 references that are in the map. First posted on arXiv 27 Feb 2026.

Builds on

29 of 34 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Multimodal Language Models See Better When They Look Shallower30 Apr
  2. 2024The Revolution of Multimodal Large Language Models: A Survey19 Feb
  3. 2023OneLLM: One Framework to Align All Modalities with Language6 Dec
  4. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
  5. 2023A Survey on Multimodal Large Language Models23 Jun
  6. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun

Other references · 23

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  4. 2019Towards VQA Models That Can Readcited by 1,696
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  6. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  8. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  9. 2025Qwen2.5-Omni Technical Reportcited by 704
  10. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  11. 2025Qwen3-Omni Technical Reportcited by 451
  12. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
11 more
  1. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  2. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
  3. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  4. 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Trainingcited by 49
  5. 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countriescited by 40
  6. 2024SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languagescited by 35
  7. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
  8. 2025Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Modelscited by 8
  9. 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 8
  10. 2025Uncovering Cultural Representation Disparities in Vision-Language Modelscited by 4
  11. 2024Compass: Large Multilingual Language Model for South-east Asiacited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
  2. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
  3. 2025Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception
  4. 2026Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs
  5. 2025OmniDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination
  6. 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination
  7. 2024OmniBench: Towards The Future of Universal Omni-Language Models
  8. 2026PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models