mlsanityData to 14 Oct 2026

Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaboration

arXiv 2504.04915 ↗First posted 7 Apr 2025cs.CL cs.AI cs.IRcited by 20

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Retrieval-Augmented Generation (RAG) systems often struggle to handle multi-hop question-answering tasks accurately due to irrelevant context retrieval and limited complex reasoning capabilities. We introduce Collab-RAG, a collaborative training framework that leverages mutual enhancement between a white-box small language model (SLM) and a blackbox large language model (LLM) for RAG. Specifically, the SLM decomposes complex queries into simpler sub-questions, thus enhancing the accuracy of the retrieval and facilitating more effective reasoning by the black-box LLM. Concurrently, the black-box LLM provides feedback signals to improve the SLM's decomposition capability. We observe that Collab-RAG relies solely on supervision from an affordable black-box LLM without additional distillation from frontier LLMs, yet demonstrates strong generalization across multiple black-box LLMs. Experimental evaluations across five multi-hop QA datasets demonstrate that Collab-RAG substantially outperforms existing black-box-only and SLM fine-tuning baselines by 1.8%-14.2% on average. In particular, our fine-tuned 3B SLM surpasses a frozen 32B LLM in question decomposition, highlighting the efficiency of Collab-RAG in improving reasoning and retrieval for complex questions. The code of Collab-RAG is available on https://github.com/ritaranx/Collab-RAG/.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 61 of its 68 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

61 of 68 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning12 Mar
  2. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
  3. 2024RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement17 Dec
  4. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions20 Dec

Other references · 57

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  9. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  10. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  12. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
38 more
  1. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  2. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  3. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
  4. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  5. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  6. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  7. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  8. 2023Active Retrieval Augmented Generationcited by 516
  9. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  10. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  12. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
  13. 2024Iterative Reasoning Preference Optimizationcited by 246
  14. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 229
  15. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
  16. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 202
  17. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 195
  18. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 184
  19. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 147
  20. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 118
  21. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
  22. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  23. 2023Improving Language Models via Plug-and-Play Retrieval Feedbackcited by 87
  24. 2024EHRAgent: Code Empowers Large Language Models for Few-shot Complex Tabular Reasoning on Electronic Health Recordscited by 78
  25. 2024Inference Scaling for Long-Context Retrieval Augmented Generationcited by 65
  26. 2024DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language Modelscited by 63
  27. 2024HYDRA: Model Factorization Framework for Black-Box LLM Personalizationcited by 60
  28. 2023ConvGQR: Generative Query Reformulation for Conversational Searchcited by 59
  29. 2024Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answeringcited by 51
  30. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAGcited by 49
  31. 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedbackcited by 46
  32. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
  33. 2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Frameworkcited by 39
  34. 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
  35. 2022Is a Question Decomposition Unit All We Need?cited by 37
  36. 2024MedAdapter: Efficient Test-Time Adaptation of Large Language Models towards Medical Reasoningcited by 36
  37. 2024BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filteringcited by 29
  38. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 8.

20 papers.

Sort:

By area:

  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  2. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  3. 2025A Survey on Collaborative Mechanisms Between Large and Small Language Modelscited by 14
  4. 2025AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Playcited by 10
  5. 2025From LLM-anation to LLM-orchestrator: Coordinating Small Models for Data Labelingcited by 8
  6. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 7
  7. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucinationcited by 5
  8. 2025FrugalRAG: Less is More in RL Finetuning for Multi-Hop Question Answeringcited by 4
  9. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  10. 2025From Evidence to Trajectory: Abductive Reasoning Path Synthesis for Retrieval-Augmented Generation Agents Developmentcited by 2
  11. 2025ADRA-Bank: A Modular Benchmark for Academic Deep Research Agentscited by 2
  12. 2025CID-GraphRAG: Enhancing Multi-Turn Dialogue Systems through Dual-Pathway Retrieval of Conversation Flow and Context Semanticscited by 1
8 more
  1. 2025Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysiscited by 1
  2. 2025MCCE: A Framework for Multi-LLM Collaborative Search in Discrete Spaces with Similarity-Filtered Preference Learningcited by 1
  3. 2025Optimizing Retrieval for RAG via Reinforcement Learningcited by 1
  4. 2026RoutIR: Fast Serving of Retrieval Pipelines for Retrieval-Augmented Generationcited by 1
  5. 2026An Agentic AI Framework with Large Language Models and Chain-of-Thought for UAV-Assisted Logistics Scheduling with Mobile Edge Computingcited by 1
  6. 2026Coordinated Semantic Alignment and Evidence Constraints for Retrieval-Augmented Generation with Large Language Modelscited by 0
  7. 2026RealRoute: Dynamic Query Routing System via Retrieve-then-Verify Paradigmcited by 0
  8. 2026Analyzing performance of Retrieval Augmented Generation models for Question Answering Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers
  2. 2026CompactRAG: Reducing LLM Calls and Token Overhead in Multi-Hop Question Answering
  3. 2025Transforming Questions and Documents for Semantically Aligned Retrieval-Augmented Generation
  4. 2025ComposeRAG: A Modular and Composable RAG for Corpus-Grounded Multi-Hop Question Answering
  5. 2026Reasoning in Trees: Improving Retrieval-Augmented Generation for Multi-Hop Question Answering
  6. 2025Question Decomposition for Retrieval-Augmented Generation
  7. 2026ConRAG: Consensus-Driven Multi-View Retrieval for Multi-Hop Question Answering
  8. 2024W-RAG: Weakly Supervised Dense Retrieval in RAG for Open-domain Question Answering