mlsanityData to 14 Oct 2026

Little Brains, Big Feats: Exploring Compact Language Models

arXiv 2606.30062 ↗First posted 29 Jun 2026cs.CL cs.AIcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

While large language models have been dominating the research landscape recently, small language models remain highly relevant across various domains; yet, they receive far less attention. In this study, we investigate how smaller language models perform during the generation stage within a Retrieval-Augmented Generation (RAG) system. To benchmark these models effectively, we utilised both open-source and proprietary datasets covering diverse subject areas and question types. Our findings demonstrate that a RAG system with small language models can be executed directly on-device without requiring any GPU hardware within a reasonable time. The experimental code and links to the supplementary materials can be accessed through the GitHub repository: https://github.com/SibNN/SLM-RAG-EVAL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 29 of its 41 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

29 of 41 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025DRAGOn: Designing RAG On Periodically Updated Corpus8 Jul
  2. 2025REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmark17 Feb
  3. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems25 Jun
  4. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries27 Jan
  5. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 24

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  9. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  10. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  11. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  12. 2024A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthinesscited by 127
12 more
  1. 2024LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domaincited by 118
  2. 2025MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systemscited by 51
  3. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 48
  4. 2025Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Modelscited by 46
  5. 2019SberQuAD -- Russian Reading Comprehension Dataset: Description and Analysiscited by 38
  6. 2024TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computerscited by 9
  7. 2020DaNetQA: a yes/no Question Answering Dataset for the Russian Languagecited by 8
  8. 2025SLM-Bench: A Comprehensive Benchmark of Small Language Models on Environmental Impacts--Extended Versioncited by 3
  9. 2025HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoringcited by 3
  10. 2026RaguTeam at SemEval-2026 Task 8: Meno and Friends in a Judge-Orchestrated LLM Ensemble for Faithful Multi-Turn Response Generationcited by 3
  11. 2024SHAKTI: A 2.5 Billion Parameter Small Language Model Optimized for Edge AI and Low-Resource Environmentscited by 2
  12. 2025SLMQuant:Benchmarking Small Language Model Quantization for Practical Deploymentcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Survey
  2. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs
  3. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Models
  4. 2024RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation
  5. 2025Less LLM, More Documents: Searching for Improved RAG
  6. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey
  7. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
  8. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning