mlsanityData to 7 Oct 2026

Fine-Tuning LLaMA for Multi-Stage Text Retrieval

arXiv 2310.08319 ↗First posted 12 Oct 2023cs.IRcited by 271

Not on the map. Why?

Cited by 56 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The effectiveness of multi-stage text retrieval has been solidly demonstrated since before the era of pre-trained language models. However, most existing studies utilize models that predate recent advances in large language models (LLMs). This study seeks to explore potential improvements that state-of-the-art LLMs can bring. We conduct a comprehensive study, fine-tuning the latest LLaMA model both as a dense retriever (RepLLaMA) and as a pointwise reranker (RankLLaMA) for both passage retrieval and document retrieval using the MS MARCO datasets. Our findings demonstrate that the effectiveness of large language models indeed surpasses that of smaller models. Additionally, since LLMs can inherently handle longer contexts, they can represent entire documents holistically, obviating the need for traditional segmenting and pooling strategies. Furthermore, evaluations on BEIR demonstrate that our RepLLaMA-RankLLaMA pipeline exhibits strong zero-shot effectiveness. Model checkpoints from this study are available on HuggingFace.

Key facts. Cited by 271 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Oct 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 92; 2025: 112; 2026: 61.

271 papers.

Sort:

By area:

  1. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  2. 2023Improving Text Embeddings with Large Language Modelscited by 295
  3. 2024Generative Representational Instruction Tuningcited by 227
  4. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  5. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  6. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 183
  7. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  8. 2024Searching for Best Practices in Retrieval-Augmented Generationcited by 83
  9. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  10. 2024Making Text Embedders Few-Shot Learnerscited by 78
  11. 2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 77
  12. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
38 more
  1. 2025ReasonIR: Training Retrievers for Reasoning Taskscited by 76
  2. 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
  3. 2024Repetition Improves Language Model Embeddingscited by 70
  4. 2025Vision-Language Models Do Not Understand Negationcited by 69
  5. 2024FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructionscited by 67
  6. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  7. 2025Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learningcited by 58
  8. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 49
  9. 2024Don't Forget to Connect! Improving RAG with Graph-based Rerankingcited by 49
  10. 2025Chain-of-Retrieval Augmented Generationcited by 48
  11. 2024Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Modelscited by 39
  12. 2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
  13. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Rerankingcited by 34
  14. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generationcited by 34
  15. 2024FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Modelscited by 33
  16. 2024INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Modelscited by 32
  17. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  18. 2025A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrievalcited by 31
  19. 2024CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Searchcited by 29
  20. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answeringcited by 28
  21. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  22. 2025RaDeR: Reasoning-aware Dense Retrieval Modelscited by 27
  23. 2024MEXA: Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignmentcited by 25
  24. 2025Diffusion vs. Autoregressive Language Models: A Text Embedding Perspectivecited by 23
  25. 2024Efficient Knowledge Injection in LLMs via Self-Distillationcited by 22
  26. 2024Natural Language Processing in the Patent Domain: A Surveycited by 20
  27. 2024Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Modelscited by 20
  28. 2024Improving Model Evaluation using SMART Filtering of Benchmark Datasetscited by 20
  29. 2025SkillFlow: Scalable and Efficient Agent Skill Retrieval Systemcited by 20
  30. 2024INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuningcited by 19
  31. 2024Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessmentcited by 19
  32. 2024Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?cited by 19
  33. 2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
  34. 2025Kanana: Compute-efficient Bilingual Language Modelscited by 19
  35. 2024Query Performance Prediction using Relevance Judgments Generated by Large Language Modelscited by 18
  36. 2024ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrievalcited by 18
  37. 2024CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generationcited by 18
  38. 2025Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systemscited by 18

50 most cited of 271 All 271 (JSON)