# A Survey of Model Architectures in Information Retrieval

arXiv: https://arxiv.org/abs/2502.14822 · HTML: https://mlsanity.com/paper/2502.14822/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2025.

## Built on by (most cited first, 20 of 20)

- [Reinforcement Learning for Self-Improving Agent with Skill Library](https://mlsanity.com/paper/2512.17102/) (2025)
- [A Comprehensive Survey on Reinforcement Learning-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications](https://mlsanity.com/paper/2510.16724/) (2025)
- [CSPLADE: Learned Sparse Retrieval with Causal Language Models](https://mlsanity.com/paper/2504.10816/) (2025)
- [ToolDreamer: Instilling LLM Reasoning Into Tool Retrievers](https://mlsanity.com/paper/2510.19791/) (2025)
- [Distillation versus Contrastive Learning: How to Train Your Rerankers](https://mlsanity.com/paper/2507.08336/) (2025)
- [Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.13272/) (2025)
- [LACONIC: Dense-Level Effectiveness for Scalable Sparse Retrieval via a Two-Phase Training Curriculum](https://mlsanity.com/paper/2601.01684/) (2026)
- [Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search](https://mlsanity.com/paper/2508.01405/) (2025)
- [Medical Triage as Pairwise Ranking: A Benchmark for Urgency in Patient Portal Messages](https://mlsanity.com/paper/2601.13178/) (2026)
- [RECON: Reasoning with Condensation for Efficient Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.10448/) (2025)
- [Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey](https://mlsanity.com/paper/2601.03262/) (2025)
- [You Only Forward Once: An Efficient Compositional Judging Paradigm](https://mlsanity.com/paper/2511.16600/) (2025)
- [An Efficient and Privacy-Preserving Architecture for Cross-Institutional Collaborative RAG](https://mlsanity.com/paper/2605.25716/) (2026)
- [MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A](https://mlsanity.com/paper/2606.04231/) (2026)
- [Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget](https://mlsanity.com/paper/2608.00916/) (2026)
- [CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflection](https://mlsanity.com/paper/2604.07487/) (2026)
- [Learning to Route Queries to Heads for Attention-based Re-ranking with Large Language Models](https://mlsanity.com/paper/2604.24608/) (2026)
- [Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers](https://mlsanity.com/paper/2608.08809/) (2026)
- [Embedding Models for Stance-Aware Argument Retrieval](https://mlsanity.com/paper/2608.28283/) (2026)
- [VerTox: Verifiable Reward-Guided Corpus Poisoning Against Neural Ranking Models](https://mlsanity.com/paper/2609.01325/) (2026)

## Abstract (arXiv)

The period from 2019 to the present marks one of the most significant paradigm shifts in information retrieval (IR) and natural language processing (NLP), culminating in the emergence of powerful large language models (LLMs) from 2022 onward. Methods based on pretrained encoder-only architectures (e.g., BERT) as well as decoder-only generative LLMs have outperformed many earlier approaches, demonstrating particularly strong performance in zero-shot scenarios and complex reasoning tasks. This survey examines the evolution of model architectures in IR, with a focus on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. To maintain analytical clarity, we deliberately separate architectural design from training methodologies, enabling a focused examination of structural innovations in IR systems. We trace the progression from traditional term-based retrieval models to modern neural approaches, highlighting the transformative impact of transformer-based architectures and subsequent LLM developments. The survey concludes with a forward-looking discussion of open challenges and emerging research directions, including architectural optimization for efficiency and scalability, robust handling of multimodal and multilingual data, and adaptation to novel application domains such as autonomous search agents, which may represent the next paradigm in IR.