Foundations of GenIR
Not on the map. Why?
Abstract (arXiv)
The chapter discusses the foundational impact of modern generative AI models on information access (IA) systems. In contrast to traditional AI, the large-scale training and superior data modeling of generative AI models enable them to produce high-quality, human-like responses, which brings brand new opportunities for the development of IA paradigms. In this chapter, we identify and introduce two of them in details, i.e., information generation and information synthesis. Information generation allows AI to create tailored content addressing user needs directly, enhancing user experience with immediate, relevant outputs. Information synthesis leverages the ability of generative AI to integrate and reorganize existing information, providing grounded responses and mitigating issues like model hallucination, which is particularly valuable in scenarios requiring precision and external knowledge. This chapter delves into the foundational aspects of generative models, including architecture, scaling, and training, and discusses their applications in multi-modal scenarios. Additionally, it examines the retrieval-augmented generation paradigm and other methods for corpus modeling and understanding, demonstrating how generative AI can enhance information access systems. It also summarizes potential challenges and fruitful directions for future studies.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 123 of its 148 references that are in the map. First posted on arXiv 6 Jan 2025.
Builds on
123 of 148 references.
References · 123
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
38 more
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2023A Survey of Large Language Modelscited by 2,576
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
- 2020Reformer: The Efficient Transformercited by 1,668
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2019Generating Long Sequences with Sparse Transformerscited by 1,546
- 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
- 2021Improving language models by retrieving from trillions of tokenscited by 1,293
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
- 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
- 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
50 most cited of 123 All 123 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026UXR PoV for Neuroinclusive Emotion Regulationcited by 0
- 2026UXR PoV for Neuroinclusive Emotion Regulation29 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Information Retrieval in the Age of Generative AI: The RGB Model
- 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Future
- 2024Exploring AI Text Generation, Retrieval-Augmented Generation, and Detection Technologies: a Comprehensive Overview
- 2026Generative AI-Based Virtual Assistant using Retrieval-Augmented Generation: An evaluation study for bachelor projects
- 2024Generative AI Systems: A Systems-based Perspective on Generative AI
- 2023A Study on the Implementation of Generative AI Services Using an Enterprise Data-Based LLM Application Architecture
- 2025If generative AI is the answer, what is the question?
- 2024Generative Information Retrieval Evaluation