mlsanityData to 7 Oct 2026

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs

arXiv 2402.12052 ↗First posted 19 Feb 2024cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 60 of its 77 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

60 of 77 references.

References · 60

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2017On Calibration of Modern Neural Networkscited by 4,860
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
38 more
  1. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  2. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  4. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  7. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  12. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  13. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  14. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  15. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  16. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  17. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  18. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  19. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  20. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  21. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  22. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  23. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  24. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  25. 2019ELI5: Long Form Question Answeringcited by 567
  26. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  27. 2023The Falcon Series of Open Language Modelscited by 526
  28. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  29. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  30. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  31. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  32. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 343
  33. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  34. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  35. 2023Tool Learning with Foundation Modelscited by 302
  36. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  37. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  38. 2020How Context Affects Language Models' Factual Predictionscited by 181

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 21; 2026: 1.

34 papers.

Sort:

By area:

  1. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  2. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  3. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  4. 2024From Matching to Generation: A Survey on Generative Information Retrievalcited by 89
  5. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  6. 2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
  7. 2024Teola: Towards End-to-End Optimization of LLM-based Applicationscited by 32
  8. 2025SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Modelcited by 31
  9. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  10. 2024RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generationcited by 20
  11. 2025DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answeringcited by 20
  12. 2024SmartRAG: Jointly Learn RAG-Related Tasks From the Environment Feedbackcited by 19
22 more
  1. 2025Hierarchical Document Refinement for Long-context Retrieval-augmented Generationcited by 16
  2. 2024Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorizationcited by 15
  3. 2025Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generationcited by 9
  4. 2025C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generationcited by 9
  5. 2025ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checkingcited by 9
  6. 2024CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generationcited by 8
  7. 2025A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthinesscited by 5
  8. 2024One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Modelscited by 4
  9. 2025Neuro-Symbolic Query Compilercited by 4
  10. 2025GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generationcited by 3
  11. 2025Learning to Route: A Rule-Driven Agent Framework for Hybrid-Source Retrieval-Augmented Generationcited by 3
  12. 2025PaperAsk: A Benchmark for Reliability Evaluation of LLMs in Paper Search and Readingcited by 3
  13. 2025KAG-Thinker: Interactive Thinking and Deep Reasoning in LLMs via Knowledge-Augmented Generationcited by 2
  14. 2025MARS: Co-evolving Dual-System Deep Research via Multi-Agent Reinforcement Learningcited by 2
  15. 2025Oreo: A Plug-in Context Reconstructor to Enhance Retrieval-Augmented Generationcited by 1
  16. 2025Single LLM, Multiple Roles: A Unified Retrieval-Augmented Generation Framework Using Role-Specific Token Optimizationcited by 1
  17. 2025Chain or tree? Re-evaluating complex reasoning from the perspective of a matrix of thoughtcited by 1
  18. 2025KCM: KAN-Based Collaboration Models Enhance Pretrained Large Modelscited by 1
  19. 2024Let your LLM generate a few tokens and you will reduce the need for retrievalcited by 0
  20. 2025Resource-Friendly Dynamic Enhancement Chain for Multi-Hop Question Answeringcited by 0
  21. 2025Error-Aware Curriculum Learning for Biomedical Relation Classificationcited by 0
  22. 2026Training-Free Test-Time Contrastive Learning for Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text
  2. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
  3. 2025Comparing Knowledge Injection Methods for LLMs in a Low-Resource Regime
  4. 2026Beyond Questions: Evaluating LLM's Knowledge Expression
  5. 2024KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models
  6. 2026RPO-RAG: Aligning Small LLMs with Relation-aware Preference Optimization for Knowledge Graph Question Answering
  7. 2025Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Reranking
  8. 2026How Much Reasoning Do Retrieval-Augmented Models Add beyond LLMs? A Benchmarking Framework for Multi-Hop Inference over Hybrid Knowledge