Data to 14 Oct 2026

FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Research

arXiv 2405.13576 ↗First posted 22 May 2024cs.CL cs.IRcited by 166

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

also in 1 more

Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

With the advent of large language models (LLMs) and multimodal large language models (MLLMs), the potential of retrieval-augmented generation (RAG) has attracted considerable research attention. Various novel algorithms and models have been introduced to enhance different aspects of RAG systems. However, the absence of a standardized framework for implementation, coupled with the inherently complex RAG process, makes it challenging and time-consuming for researchers to compare and evaluate these approaches in a consistent environment. Existing RAG toolkits, such as LangChain and LlamaIndex, while available, are often heavy and inflexibly, failing to meet the customization needs of researchers. In response to this challenge, we develop ours, an efficient and modular open-source toolkit designed to assist researchers in reproducing and comparing existing RAG methods and developing their own algorithms within a unified framework. Our toolkit has implemented 16 advanced RAG methods and gathered and organized 38 benchmark datasets. It has various features, including a customizable modular framework, multimodal RAG capabilities, a rich collection of pre-implemented RAG works, comprehensive datasets, efficient auxiliary pre-processing scripts, and extensive and standard evaluation metrics. Our toolkit and resources are available at https://github.com/RUC-NLPIR/FlashRAG.

Key facts. Cited by 166 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 74 of its 111 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

74 of 111 references.

References · 74

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
2020Measuring Massive Multitask Language Understandingcited by 7,963
2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955

38 more

2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
2017Billion-scale similarity search with GPUscited by 2,497
2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
2021Improving language models by retrieving from trillions of tokenscited by 1,293
2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
2020Aligning AI With Shared Human Valuescited by 723
2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
2017The NarrativeQA Reading Comprehension Challengecited by 597
2019ELI5: Long Form Question Answeringcited by 575
2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
2023Active Retrieval Augmented Generationcited by 516
2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
2024The Faiss librarycited by 376
2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 19; 2025: 84; 2026: 63.

166 papers.

Sort:

By area:

2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
2025Agentic Reinforced Policy Optimizationcited by 172
2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 161
2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 101
2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 72
2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 70
2025HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representationcited by 68
2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 52
2025Chain-of-Retrieval Augmented Generationcited by 49
2024Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Modelscited by 43
2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
2025Agentic Entropy-Balanced Policy Optimizationcited by 40

38 more

2024RAGEval: Scenario Specific RAG Evaluation Dataset Generation Frameworkcited by 39
2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
2025R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learningcited by 34
2024Ragnar"ok: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Trackcited by 32
2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewardingcited by 32
2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
2025R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learningcited by 30
2025Removal of Hallucination on Hallucination: Debate-Augmented RAGcited by 29
2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
2024Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systemscited by 26
2024RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructionscited by 25
2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 25
2025Knowing You Don't Know: Learning When to Continue Search in Multi-round RAG through Self-Practicingcited by 25
2025SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reductioncited by 24
2025DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AIcited by 23
2024RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generationcited by 20
2025DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answeringcited by 20
2025Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foragingcited by 20
2025R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learningcited by 20
2026MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baselinecited by 20
2025LexRAG: Benchmarking Retrieval-Augmented Generation in Multi-Turn Legal Consultation Conversationcited by 19
2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 18
2025Agent models: Internalizing Chain-of-Action Generation into Reasoning modelscited by 17
2026Adaptive Rollout Allocation for Online Reinforcement Learning with Verifiable Rewardscited by 17
2024Ward: Provable RAG Dataset Inference via LLM Watermarkscited by 16
2025Hierarchical Document Refinement for Long-context Retrieval-augmented Generationcited by 16
2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 15
2025PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluationcited by 14
2026SmartSearch: Process Reward-Guided Query Refinement for Search Agentscited by 14
2024Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generationcited by 13
2025Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generationcited by 13
2025KiRAG: Knowledge-Driven Iterative Retriever for Enhancing Retrieval-Augmented Generationcited by 13
2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
2025LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searcherscited by 11
2025QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errorscited by 11
2025ComposeRAG: A Modular and Composable RAG for Corpus-Grounded Multi-Hop Question Answeringcited by 11
2026Over-Searching in Search-Augmented Large Language Modelscited by 11

50 most cited of 166 All 166 (JSON)

Similar, not cited

Close in wording, no citation either way