mlsanityData to 14 Oct 2026

Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation

arXiv 2609.16059 ↗First posted 13 Sep 2026cs.CL cs.LGcited by 0

On the map: Synthetic data and self-improvement for LLMs → Self-play RL for reasoning

Abstract (arXiv)

Multimodal instruction following (MMIF) is crucial for building generalist agents. However, current training paradigms rely heavily on Supervised Fine-Tuning (SFT), which often leads to surface-level pattern matching and degrades general capabilities. While Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising alternative, its scalability in MMIF is severely bottlenecked by the scarcity of high-quality, RL-ready multimodal data. To bridge this gap, we present MIFS (Multimodal Instruction Following Synthesis), a systematic pipeline designed to generate RL-ready multimodal data. Specifically, MIFS introduces a generative constraint protocol to synthesize diverse raw samples, followed by a learnability-aware distillation mechanism that filters data based on RL training dynamics to ensure stable policy optimization. Furthermore, a code-based verifier provides high-precision reward signals for policy learning. The resulting dataset comprises 90k samples across 8 constraint categories and 14 task domains. Empirical evaluations demonstrate that MIFS-trained MLLMs achieve an average improvement of 8.13% on four MMIF benchmarks and a 3× faster training convergence compared to using raw data. Crucially, our approach mitigates the generalization trade-offs typical of SFT, preserving core visual capabilities while significantly boosting instruction-following precision.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Self-play RL for reasoning, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 22 of its 31 references that are in the map. First posted on arXiv 13 Sep 2026.

Builds on

22 of 31 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  4. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  6. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  7. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  8. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  9. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 186
  10. 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 90
  11. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 70
  12. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 67
10 more
  1. 2024MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMscited by 62
  2. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 56
  3. 2025VerIF: Verification Engineering for Reinforcement Learning in Instruction Followingcited by 44
  4. 2025MM-IFEngine: Towards Multimodal Instruction Followingcited by 42
  5. 2024MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversitycited by 38
  6. 2025OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preferencecited by 28
  7. 2023TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wildcited by 22
  8. 2025CrafText Benchmark: Advancing Instruction Following in Complex Multimodal Open-Ended Worldcited by 2
  9. 2026AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraintscited by 2
  10. 2026Precision over Diversity: High-Precision Reward Generalizes to Robust Instruction Followingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning
  2. 2026Improving Generalization Robustness of Multimodal RLVR
  3. 2025Stable and Efficient Single-Rollout RL for Multimodal Reasoning
  4. 2026MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning
  5. 2026SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
  6. 2026Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidance
  7. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources
  8. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models