mlsanityData to 14 Oct 2026

Foundations of GenIR

arXiv 2501.02842 ↗First posted 6 Jan 2025cs.IR cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

The chapter discusses the foundational impact of modern generative AI models on information access (IA) systems. In contrast to traditional AI, the large-scale training and superior data modeling of generative AI models enable them to produce high-quality, human-like responses, which brings brand new opportunities for the development of IA paradigms. In this chapter, we identify and introduce two of them in details, i.e., information generation and information synthesis. Information generation allows AI to create tailored content addressing user needs directly, enhancing user experience with immediate, relevant outputs. Information synthesis leverages the ability of generative AI to integrate and reorganize existing information, providing grounded responses and mitigating issues like model hallucination, which is particularly valuable in scenarios requiring precision and external knowledge. This chapter delves into the foundational aspects of generative models, including architecture, scaling, and training, and discusses their applications in multi-modal scenarios. Additionally, it examines the retrieval-augmented generation paradigm and other methods for corpus modeling and understanding, demonstrating how generative AI can enhance information access systems. It also summarizes potential challenges and fruitful directions for future studies.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 123 of its 148 references that are in the map. First posted on arXiv 6 Jan 2025.

Builds on

123 of 148 references.

References · 123

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2017Proximal Policy Optimization Algorithmscited by 15,215
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  9. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  10. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  12. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
38 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  13. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  14. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  15. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  16. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  17. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  18. 2023A Survey of Large Language Modelscited by 2,576
  19. 2022Emergent Abilities of Large Language Modelscited by 2,512
  20. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  21. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  22. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  23. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  24. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  25. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  26. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  27. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  28. 2020Reformer: The Efficient Transformercited by 1,668
  29. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  30. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  31. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  32. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  33. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  34. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  35. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  36. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  37. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  38. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045

50 most cited of 123 All 123 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Information Retrieval in the Age of Generative AI: The RGB Model
  2. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Future
  3. 2024Exploring AI Text Generation, Retrieval-Augmented Generation, and Detection Technologies: a Comprehensive Overview
  4. 2026Generative AI-Based Virtual Assistant using Retrieval-Augmented Generation: An evaluation study for bachelor projects
  5. 2024Generative AI Systems: A Systems-based Perspective on Generative AI
  6. 2023A Study on the Implementation of Generative AI Services Using an Enterprise Data-Based LLM Application Architecture
  7. 2025If generative AI is the answer, what is the question?
  8. 2024Generative Information Retrieval Evaluation