A Curriculum Learning Approach to Reinforcement Learning: Leveraging RAG for Multimodal Question Answering
On the map: Deep reinforcement learning methods → RL training of language and agent models
Abstract (arXiv)
This paper describes the solutions of the Dianping-Trust-Safety team for the META CRAG-MM challenge. The challenge requires building a comprehensive retrieval-augmented generation system capable for multi-modal multi-turn question answering. The competition consists of three tasks: (1) answering questions using structured data retrieved from an image-based mock knowledge graph, (2) synthesizing information from both knowledge graphs and web search results, and (3) handling multi-turn conversations that require context understanding and information aggregation from multiple sources. For Task 1, our solution is based on the vision large language model, enhanced by supervised fine-tuning with knowledge distilled from GPT-4.1. We further applied curriculum learning strategies to guide reinforcement learning, resulting in improved answer accuracy and reduced hallucination. For Task 2 and Task 3, we additionally leveraged web search APIs to incorporate external knowledge, enabling the system to better handle complex queries and multi-turn conversations. Our approach achieved 1st place in Task 1 with a significant lead of 52.38%, and 3rd place in Task 3, demonstrating the effectiveness of the integration of curriculum learning with reinforcement learning in our training pipeline.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 21 references that are in the map. First posted on arXiv 14 Aug 2025.
Builds on
12 of 21 references.
Closest ancestors · 1
Same direction, newest first
- 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments4 Apr
Other references · 11
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015VQA: Visual Question Answeringcited by 2,506
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
- 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
- 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
- 2015Ask Your Neurons: A Neural-based Approach to Answering Questions about Imagescited by 51
- 2015Learning to Answer Questions From Image Using Convolutional Neural Networkcited by 23
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Curriculum Guided Reinforcement Learning for Efficient Multi Hop Retrieval Augmented Generation
- 2024Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning
- 2025ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answering
- 2025Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training
- 2026PRISMA: Reinforcement Learning Guided Two-Stage Policy Optimization in Multi-Agent Architecture for Open-Domain Multi-Hop Question Answering
- 2026MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
- 2025ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering
- 2025UR2: Unify RAG and Reasoning through Reinforcement Learning