How RL Unlocks the Aha Moment in Geometric Interleaved Reasoning
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Solving complex geometric problems inherently requires interleaved reasoning: a tight alternation between constructing diagrams and performing logical deductions. Although recent Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities in visual generation and plotting, we identify a counter-intuitive and underexplored phenomenon. Naively applying Supervised Fine-Tuning (SFT) on interleaved plot-solution data leads to a substantial degradation in reasoning performance compared to text-only baselines. We argue that this failure stems from a fundamental limitation of SFT, which primarily induces distributional alignment: the model learns to reproduce the surface format of interleaved plotting but fails to internalize the causal dependency between the generated plot and reasoning steps. To overcome this limitation, we propose Faire (Functional alignment for interleaved reasoning), a reinforcement learning framework that enforces three casual constraints to move beyond superficial imitation toward functional alignment. Extensive experiments show that Faire induces a qualitative shift in model behavior in which the plotting is effectively internalized, yielding competitive performance on challenging geometric reasoning benchmarks.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 78 references that are in the map. First posted on arXiv 1 Mar 2026.
Builds on
25 of 78 references.
Closest ancestors · 3
Same direction, newest first
- 2025Geoint-R1: Formalizing Multimodal Geometric Reasoning with Dynamic Auxiliary Constructions5 Aug
- 2025NeSyGeo: A Neuro-Symbolic Framework for Multimodal Geometric Reasoning Data Generation21 May
- 2025TrustGeoGen: Formal-Verified Data Engine for Trustworthy Multi-modal Geometric Problem Solving22 Apr
Other references · 22
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
- 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
- 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 55
- 2025Interleaving Reasoning for Better Text-to-Image Generationcited by 40
10 more
- 2024MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classificationcited by 35
- 2024Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmarkcited by 22
- 2025GenExam: A Multidisciplinary Text-to-Image Examcited by 19
- 2025SOLIDGEO: Measuring Multimodal Spatial Math Reasoning in Solid Geometrycited by 16
- 2025ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoningcited by 13
- 2025MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problemscited by 10
- 2025GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Modelscited by 6
- 2024MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contextscited by 5
- 2025Generating Pedagogically Meaningful Visuals for Math Word Problems: A New Benchmark and Analysis of Text-to-Image Modelscited by 4
- 2025DiagramIR: An Automatic Pipeline for Educational Math Diagram Evaluationcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Thinking with Constructions: A Benchmark and Policy Optimization for Visual-Text Interleaved Geometric Reasoning
- 2024EAGLE: Elevating Geometric Reasoning through LLM-empowered Visual Instruction Tuning
- 2026GeoMathCode: Understanding Interleaved Math-Code Reasoning for Geometry Problem Solving
- 2025Bridging Formal Language with Chain-of-Thought Reasoning to Geometry Problem Solving
- 2024Beyond Lines and Circles: Unveiling the Geometric Reasoning Gap in Large Language Models
- 2026MetaReason: Precise Interleaved Multimodal Reasoning via Editing Meta Information for Solving Geometry Problems
- 2024GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training
- 2025Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasoner