mlsanityData to 14 Oct 2026

Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach

arXiv 2602.13890 ↗First posted 14 Feb 2026cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

Retrieval Augmented Generation (RAG) is a powerful approach for enhancing the factual grounding of language models by integrating external knowledge. While widely studied for large language models, the optimization of RAG for Small Language Models (SLMs) remains a critical research gap, particularly in complex, multi-hop question-answering tasks that require sophisticated reasoning. In these systems, prompt template design is a crucial yet under-explored factor influencing performance. This paper presents a large-scale empirical study to investigate this factor, evaluating 24 different prompt templates on the HotpotQA dataset. The set includes a standard RAG prompt, nine well-formed techniques from the literature, and 14 novel hybrid variants, all tested on two prominent SLMs: Qwen2.5-3B Instruct and Gemma3-4B-It. Our findings, based on a test set of 18720 instances, reveal significant performance gains of up to 83% on Qwen2.5 and 84.5% on Gemma3-4B-It, yielding an improvement of up to 6% for both models compared to the Standard RAG prompt. This research also offers concrete analysis and actionable recommendations for designing effective and efficient prompts for SLM-based RAG systems, practically for deployment in resource-constrained environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 13 of its 17 references that are in the map. First posted on arXiv 14 Feb 2026.

Builds on

13 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems25 Jun
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Survey29 Feb

Other references · 11

  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  3. 2024A Survey on LLM-as-a-Judgecited by 1,423
  4. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  5. 2022Teaching Small Language Models to Reasoncited by 284
  6. 2024The Prompt Report: A Systematic Survey of Prompt Engineering Techniquescited by 121
  7. 2022Multi-hop Question Answeringcited by 64
  8. 2023Prompt Engineering a Prompt Engineercited by 53
  9. 2019Multi-hop Question Answering via Reasoning Chainscited by 42
  10. 2020Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?cited by 30
  11. 2022Few-shot Reranking for Multi-hop QA via Language Model Promptingcited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Understanding the Fundamental Design Decisions of Retrieval-Augmented Generation Systems
  2. 2023Establishing Performance Baselines in Fine-Tuning, Retrieval-Augmented Generation and Soft-Prompting for Non-Specialist LLM Users
  3. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  4. 2026Evaluating Multi-Hop Reasoning in RAG Systems: A Comparison of LLM-Based Retriever Evaluation Strategies
  5. 2024A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions
  6. 2025A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions
  7. 2024The Power of Noise: Redefining Retrieval for RAG Systems
  8. 2026Little Brains, Big Feats: Exploring Compact Language Models