Unlocking Cognitive Capabilities and Analyzing the Perception-Logic Trade-off
On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Abstract (arXiv)
Recent advancements in Multimodal Large Language Models (MLLMs) pursue omni-perception capabilities, yet integrating robust sensory grounding with complex reasoning remains a challenge, particularly for underrepresented regions. In this report, we introduce the research preview of MERaLiON2-Omni (Alpha), a 10B-parameter multilingual omni-perception tailored for Southeast Asia (SEA). We present a progressive training pipeline that explicitly decouples and then integrates "System 1" (Perception) and "System 2" (Reasoning) capabilities. First, we establish a robust Perception Backbone by aligning region-specific audio-visual cues (e.g., Singlish code-switching, local cultural landmarks) with a multilingual LLM through orthogonal modality adaptation. Second, to inject cognitive capabilities without large-scale supervision, we propose a cost-effective Generate-Judge-Refine pipeline. By utilizing a Super-LLM to filter hallucinations and resolve conflicts via a consensus mechanism, we synthesize high-quality silver data that transfers textual Chain-of-Thought reasoning to multimodal scenarios. Comprehensive evaluation on our newly introduced SEA-Omni Benchmark Suite reveals an Efficiency-Stability Paradox: while reasoning acts as a non-linear amplifier for abstract tasks (boosting mathematical and instruction-following performance significantly), it introduces instability in low-level sensory processing. Specifically, we identify Temporal Drift in long-context audio, where extended reasoning desynchronizes the model from acoustic timestamps, and Visual Over-interpretation, where logic overrides pixel-level reality. This report details the architecture, the data-efficient training recipe, and a diagnostic analysis of the trade-offs between robust perception and structured reasoning.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 34 references that are in the map. First posted on arXiv 27 Feb 2026.
Builds on
29 of 34 references.
Closest ancestors · 6
Same direction, newest first
- 2025Multimodal Language Models See Better When They Look Shallower30 Apr
- 2024The Revolution of Multimodal Large Language Models: A Survey19 Feb
- 2023OneLLM: One Framework to Align All Modalities with Language6 Dec
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
- 2023A Survey on Multimodal Large Language Models23 Jun
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models23 Jun
Other references · 23
- 2025Qwen3 Technical Reportcited by 8,195
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2019Towards VQA Models That Can Readcited by 1,696
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
- 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
- 2025Qwen3-Omni Technical Reportcited by 451
- 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
11 more
- 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
- 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 124
- 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
- 20251.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Trainingcited by 49
- 2025CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countriescited by 40
- 2024SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languagescited by 35
- 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
- 2025Advancing Singlish Understanding: Bridging the Gap with Datasets and Multimodal Modelscited by 8
- 2025Crowdsource, Crawl, or Generate? Creating SEA-VL, a Multicultural Vision-Language Dataset for Southeast Asiacited by 8
- 2025Uncovering Cultural Representation Disparities in Vision-Language Modelscited by 4
- 2024Compass: Large Multilingual Language Model for South-east Asiacited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
- 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
- 2025Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception
- 2026Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs
- 2025OmniDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination
- 2024Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination
- 2024OmniBench: Towards The Future of Universal Omni-Language Models
- 2026PerceptionBench: Evaluating Atomic Visual Perception in Multimodal Large Language Models