mlsanityData to 14 Oct 2026

LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference

arXiv 2505.12260 ↗First posted 18 May 2025cs.IR cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs)-based text retrieval retrieves documents relevant to search queries based on vector similarities. Documents are pre-encoded offline, while queries arrive in real-time, necessitating an efficient online query encoder. Although LLMs significantly enhance retrieval capabilities, serving deeply parameterized LLMs slows down query inference throughput and increases demands for online deployment resources. In this paper, we propose LightRetriever, a novel LLM-based retriever with extremely lightweight query encoders. Our method retains a full-sized LLM for document encoding, but reduces the workload of query encoding to no more than an embedding lookup. Compared to serving a full LLM on an A800 GPU, our method achieves over 1000x speedup in query encoding and over 10x increase in end-to-end retrieval throughput. Extensive experiments on large-scale retrieval benchmarks show that LightRetriever generalizes well across diverse tasks, maintaining an average of 95% retrieval performance.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 59 of its 96 references that are in the map. First posted on arXiv 18 May 2025.

Builds on

59 of 96 references.

References · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  10. 2023Qwen Technical Reportcited by 3,344
  11. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
38 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  2. 2017Billion-scale similarity search with GPUscited by 2,497
  3. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  6. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  9. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  11. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  12. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  13. 2019ELI5: Long Form Question Answeringcited by 575
  14. 2020Fact or Fiction: Verifying Scientific Claimscited by 554
  15. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  16. 2021Large Dual Encoders Are Generalizable Retrieverscited by 456
  17. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 430
  18. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
  19. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  20. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 333
  21. 2023Improving Text Embeddings with Large Language Modelscited by 303
  22. 2020Overview of the TREC 2019 deep learning trackcited by 286
  23. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
  24. 2022Matryoshka Representation Learningcited by 282
  25. 2018WikiHow: A Large Scale Text Summarization Datasetcited by 204
  26. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  27. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  28. 2024BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrievalcited by 158
  29. 2024Liger Kernel: Efficient Triton Kernels for LLM Trainingcited by 130
  30. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 127
  31. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  32. 2024SPLADE-v3: New baselines for SPLADEcited by 72
  33. 2023T2Ranking: A large-scale Chinese Benchmark for Passage Rankingcited by 60
  34. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 60
  35. 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
  36. 2020SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrievalcited by 50
  37. 2020Minimizing FLOPs to Learn Efficient Sparse Representationscited by 46
  38. 2020SparTerm: Learning Term-based Sparse Representation for Fast Text Retrievalcited by 45

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Mixture-of-Experts Language Models Can Be Strong and Efficient Retrieverscited by 0