mlsanityData to 14 Oct 2026

RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts

arXiv 2502.17888 ↗First posted 25 Feb 2025cs.CLcited by 6

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Retrieval-Augmented Generation (RAG) enhances the performance of Large Language Models (LLMs) by incorporating external knowledge. However, LLMs still encounter challenges in effectively utilizing the knowledge from retrieved documents, often being misled by irrelevant or noisy information. To address this issue, we introduce RankCoT, a knowledge refinement method that incorporates reranking signals in generating CoT-based summarization for knowledge refinement based on given query and all retrieval documents. During training, RankCoT prompts the LLM to generate Chain-of-Thought (CoT) candidates based on the query and individual documents. It then fine-tunes the LLM to directly reproduce the best CoT from these candidate outputs based on all retrieved documents, which requires LLM to filter out irrelevant documents during generating CoT-style summarization. Additionally, RankCoT incorporates a self-reflection mechanism that further refines the CoT outputs, resulting in higher-quality training data. Our experiments demonstrate the effectiveness of RankCoT, showing its superior performance over other knowledge refinement models. Further analysis reveals that RankCoT can provide shorter but effective refinement results, enabling the generator to produce more accurate answers. All code and data are available at https://github.com/NEUIR/RankCoT.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 58 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

23 of 58 references.

References · 23

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  8. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  9. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  10. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  11. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  12. 2022A Survey on In-context Learningcited by 721
11 more
  1. 2023Active Retrieval Augmented Generationcited by 516
  2. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  3. 2023The False Promise of Imitating Proprietary LLMscited by 263
  4. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 202
  5. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  6. 2024Corrective Retrieval Augmented Generationcited by 197
  7. 2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
  8. 2024Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworkscited by 42
  9. 2024RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewardscited by 33
  10. 2024Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generationcited by 19
  11. 2024ThinkNote: Enhancing Knowledge Integration and Utilization of Large Language Models via Constructivist Cognition Modelingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025ReMindRAG: Low-Cost LLM-Guided Knowledge Graph Traversal for Efficient RAGcited by 8
  2. 2025Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimizationcited by 5
  3. 2025Learning to Ponder: Adaptive Reasoning in Latent Spacecited by 5
  4. 2025From Faithfulness to Correctness: Generative Reward Models that Think Criticallycited by 4
  5. 2025HetaRAG: Hybrid Deep Retrieval-Augmented Generation across Heterogeneous Data Storescited by 3
  6. 2026Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Direct Retrieval-augmented Optimization: Synergizing Knowledge Selection and Language Models
  2. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation
  3. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generation
  4. 2025Chain-of-Rank: Enhancing Large Language Models for Domain-Specific RAG in Edge Device
  5. 2026LLM-Confidence Reranker: A Training-Free Approach for Enhancing Retrieval-Augmented Generation Systems
  6. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation
  7. 2026Rethinking Retrieval-Augmented Generation as a Cooperative Decision-Making Problem
  8. 2024Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts