mlsanityData to 7 Oct 2026

OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMs

arXiv 2409.05152 ↗First posted 8 Sep 2024cs.CL cs.AI cs.DBcited by 14

Not on the map. Why?

Abstract (arXiv)

Despite the recent advancements in Large Language Models (LLMs), which have significantly enhanced the generative capabilities for various NLP tasks, LLMs still face limitations in directly handling retrieval tasks. However, many practical applications demand the seamless integration of both retrieval and generation. This paper introduces a novel and efficient One-pass Generation and retrieval framework (OneGen), designed to improve LLMs' performance on tasks that require both generation and retrieval. The proposed framework bridges the traditionally separate training approaches for generation and retrieval by incorporating retrieval tokens generated autoregressively. This enables a single LLM to handle both tasks simultaneously in a unified forward pass. We conduct experiments on two distinct types of composite tasks, RAG and Entity Linking, to validate the pluggability, effectiveness, and efficiency of OneGen in training and inference. Furthermore, our results show that integrating generation and retrieval within the same context preserves the generative capabilities of LLMs while improving retrieval performance. To the best of our knowledge, OneGen is the first to enable LLMs to conduct vector retrieval during the generation.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 59 of its 73 references that are in the map. First posted on arXiv 8 Sep 2024.

Builds on

59 of 73 references.

References · 59

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
38 more
  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  8. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  11. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  12. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  13. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  14. 2023Active Retrieval Augmented Generationcited by 506
  15. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  16. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  17. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  18. 2020Autoregressive Entity Retrievalcited by 333
  19. 2023Improving Text Embeddings with Large Language Modelscited by 295
  20. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  21. 2023Fine-Tuning LLaMA for Multi-Stage Text Retrievalcited by 271
  22. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  23. 2024Generative Representational Instruction Tuningcited by 227
  24. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  25. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  26. 2024RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generationcited by 144
  27. 2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
  28. 2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
  29. 2018End-to-End Neural Entity Linkingcited by 85
  30. 2023Scaling Sentence Embeddings with Large Language Modelscited by 83
  31. 2022ReFinED: An Efficient Zero-shot-capable Approach to End-to-End Entity Linkingcited by 74
  32. 2024Repetition Improves Language Model Embeddingscited by 70
  33. 2020REL: An Entity Linker Standing on the Shoulders of Giantscited by 61
  34. 2020Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linkingcited by 49
  35. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAGcited by 47
  36. 2023ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Modelscited by 44
  37. 2024Steering Without Side Effects: Improving Post-Deployment Control of Language Modelscited by 44
  38. 2021EntQA: Entity Linking as Question Answeringcited by 34

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 7.

14 papers.

Sort:

By area:

  1. 2025LightThinker: Thinking Step-by-Step Compressioncited by 104
  2. 2025Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answeringcited by 22
  3. 2024RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generationcited by 20
  4. 2025Large Reasoning Embedding Models: Towards Next-Generation Dense Retrieval Paradigmcited by 9
  5. 2025MAGNET: Augmenting Generative Decoders with Representation Learning and Infilling Capabilitiescited by 5
  6. 2025ImpRAG: Retrieval-Augmented Generation with Implicit Queriescited by 5
  7. 2024Large Language Model Can Be a Foundation for Hidden Rationale-Based Retrievalcited by 1
  8. 2026Hydra: Unifying Document Retrieval and Generation in a Single Vision-Language Modelcited by 0
  9. 2026A Unified Model and Document Representation for On-Device Retrieval-Augmented Generationcited by 0
  10. 2026Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPAcited by 0
  11. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  12. 2026Co-LMLM: Continuous-Query Limited Memory Language Modelscited by 0
2 more
  1. 2026Token-Native Storage: Read and Write in your Agent's Languagecited by 0
  2. 2026Select, Don't Train: The Benefits of Modular Entity Disambiguation with LLM-Based Selectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025When Retrieval Succeeds and Fails: Rethinking Retrieval-Augmented Generation for LLMs
  2. 2024Retrieval-Augmented Generation for Natural Language Processing: A Survey
  3. 2024A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential
  4. 2025Retrieval-Augmented Generation: A Comprehensive Survey of Architectures, Enhancements, and Robustness Frontiers
  5. 2024Evaluation of Retrieval-Augmented Generation: A Survey
  6. 2024A Survey on Retrieval-Augmented Text Generation for Large Language Models
  7. 2024Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
  8. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generation