mlsanityData to 14 Oct 2026

Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles

arXiv 2505.23590 ↗First posted 29 May 2025cs.CV cs.AI cs.CLcited by 36

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

The application of rule-based reinforcement learning (RL) to multimodal large language models (MLLMs) introduces unique challenges and potential deviations from findings in text-only domains, particularly for perception-heavy tasks. This paper provides a comprehensive study of rule-based visual RL, using jigsaw puzzles as a structured experimental framework. Jigsaw puzzles offer inherent ground truth, adjustable difficulty, and demand complex decision-making, making them ideal for this study. Our research reveals several key findings: textitFirstly, we find that MLLMs, initially performing near to random guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and generalize to complex, unseen configurations through fine-tuning. textitSecondly, training on jigsaw puzzles can induce generalization to other visual tasks, with effectiveness tied to specific task configurations. textitThirdly, MLLMs can learn and generalize with or without explicit reasoning, though open-source models often favor direct answering. Consequently, even when trained for step-by-step reasoning, they can ignore the thinking process in deriving the final answer. textitFourthly, we observe that complex reasoning patterns appear to be pre-existing rather than emergent, with their frequency increasing alongside training and task difficulty. textitFinally, our results demonstrate that RL exhibits more effective generalization than Supervised Fine-Tuning (SFT), and an initial SFT cold start phase can hinder subsequent RL optimization. Although these observations are based on jigsaw puzzles and may vary across other visual tasks, this research contributes a valuable piece of jigsaw to the larger puzzle of collective understanding rule-based visual RL and its potential in multimodal learning. The code is available at: https://github.com/zifuwanggg/Jigsaw-R1

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 87 of its 110 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

87 of 110 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Training28 May
  2. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
  3. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation17 Apr
  4. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
  5. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  6. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  7. 2025Improved Visual-Spatial Reasoning via R1-Zero-Like Training1 Apr
  8. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  9. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning10 Mar
  10. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models10 Dec

Other references · 77

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2023Let's Verify Step by Stepcited by 3,789
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  12. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
38 more
  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  4. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  6. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
  7. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  8. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  9. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  10. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  11. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  12. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  13. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  14. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  15. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  16. 2019Domain Generalization by Solving Jigsaw Puzzlescited by 450
  17. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  18. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  19. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  20. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
  21. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  22. 2024Titans: Learning to Memorize at Test Timecited by 328
  23. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learningcited by 303
  24. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  25. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  26. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  27. 2025Learning to Reason without External Rewardscited by 210
  28. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
  29. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  30. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 197
  31. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 182
  32. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166
  33. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 165
  34. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 150
  35. 2018Self-Supervised Video Representation Learning with Space-Time Cubic Puzzlescited by 149
  36. 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Modelscited by 144
  37. 2020Self-supervised Video Representation Learning by Pace Predictioncited by 139
  38. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 134

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 15.

36 papers.

Sort:

By area:

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  2. 2025Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Visioncited by 56
  3. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 56
  4. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  5. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 41
  6. 2025Beyond Seeing: Evaluating Multimodal LLMs on Tool-Enabled Image Perception, Transformation, and Reasoningcited by 29
  7. 2025When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thoughtcited by 21
  8. 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Modelscited by 19
  9. 2025Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspectivecited by 14
  10. 2025VR-Thinker: Boosting Video Reward Models through Thinking-with-Image Reasoningcited by 12
  11. 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11
  12. 2026VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agentscited by 11
24 more
  1. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  2. 2025Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalizationcited by 8
  3. 2025SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoningcited by 7
  4. 2025ViSS-R1: Self-Supervised Reinforcement Video Reasoningcited by 6
  5. 2026Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoningcited by 6
  6. 2025Reassessing the Role of Supervised Fine-Tuning: An Empirical Study in VLM Reasoningcited by 5
  7. 2026Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Processcited by 5
  8. 2025PuzLM: Solving Jigsaw Puzzles with Sequence-to-Sequence Language Modelscited by 4
  9. 2025PuzzleCraft: Exploration-Aware Curriculum Learning for Puzzle-Based RLVR in VLMscited by 4
  10. 2026Improving Vision-language Models with Perception-centric Process Reward Modelscited by 4
  11. 2025DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Modelcited by 3
  12. 2025SPHINX: A Synthetic Environment for Visual Perception and Reasoningcited by 2
  13. 2026Med-Scout: Curing MLLMs' Geometric Blindness in Medical Perception via Geometry-Aware RL Post-Trainingcited by 2
  14. 2026TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Modelscited by 2
  15. 2026Learning Transferable Temporal Primitives for Video Reasoning via Synthetic Videoscited by 2
  16. 2026SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Modelscited by 2
  17. 2026Boosting Visual Instruction Tuning with Self-Supervised Guidancecited by 1
  18. 2026Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoningcited by 1
  19. 2026JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzlescited by 1
  20. 2025Rule-Based Reinforcement Learning for Document Image Classification with Vision Language Modelscited by 0
  21. 2026DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differencescited by 0
  22. 2026ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Trainingcited by 0
  23. 2026JigSync: Gauge-Resolved Synchronization for Jigsaw Reassembly under Unknown Piece Orientationcited by 0
  24. 2026Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Visual Jigsaw Post-Training Improves MLLMs
  2. 2025Play to Generalize: Learning to Reason Through Game Play
  3. 2025Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning
  4. 2025Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning
  5. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
  6. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use
  7. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
  8. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learning