# Large Language Models vs. Search Engines: Evaluating User Preferences Across Varied Information Retrieval Scenarios

arXiv: https://arxiv.org/abs/2401.05761 · HTML: https://mlsanity.com/paper/2401.05761/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2024.

## Built on by (most cited first, 14 of 14)

- [Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization](https://mlsanity.com/paper/2410.09942/) (2024)
- [SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding](https://mlsanity.com/paper/2506.11309/) (2025)
- [Source Coverage and Citation Bias in LLM-based vs. Traditional Search Engines](https://mlsanity.com/paper/2512.09483/) (2025)
- [An Assessment of Model-On-Model Deception](https://mlsanity.com/paper/2405.12999/) (2024)
- [Evaluating Search Engines and Large Language Models for Answering Health Questions](https://mlsanity.com/paper/2407.12468/) (2024)
- [PQPP: A Joint Benchmark for Text-to-Image Prompt and Query Performance Prediction](https://mlsanity.com/paper/2406.04746/) (2024)
- [Agent-centric Information Access](https://mlsanity.com/paper/2502.19298/) (2025)
- [Calibrated Trust in Dealing with LLM Hallucinations: A Qualitative Study](https://mlsanity.com/paper/2512.09088/) (2025)
- [Evaluation of Agents under Simulated AI Marketplace Dynamics](https://mlsanity.com/paper/2604.14256/) (2026)
- [Can AI Agents Synthesize Scientific Conclusions?](https://mlsanity.com/paper/2606.11337/) (2026)
- [CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference](https://mlsanity.com/paper/2508.04462/) (2025)
- [Rethinking Search: A Study of University Students' Perspectives on Using LLMs and Traditional Search Engines in Academic Problem Solving](https://mlsanity.com/paper/2510.17726/) (2025)
- [AI Hallucination from Students' Perspective: A Thematic Analysis](https://mlsanity.com/paper/2602.17671/) (2026)
- [Overview of the TREC 2025 Million Large Language Models track](https://mlsanity.com/paper/2609.31921/) (2026)

## Abstract (arXiv)

This study embarked on a comprehensive exploration of user preferences between Search Engines and Large Language Models (LLMs) in the context of various information retrieval scenarios. Conducted with a sample size of 100 internet users (N=100) from across the United States, the research delved into 20 distinct use cases ranging from factual searches, such as looking up COVID-19 guidelines, to more subjective tasks, like seeking interpretations of complex concepts in layman's terms. Participants were asked to state their preference between using a traditional search engine or an LLM for each scenario. This approach allowed for a nuanced understanding of how users perceive and utilize these two predominant digital tools in differing contexts. The use cases were carefully selected to cover a broad spectrum of typical online queries, thus ensuring a comprehensive analysis of user preferences. The findings reveal intriguing patterns in user choices, highlighting a clear tendency for participants to favor search engines for direct, fact-based queries, while LLMs were more often preferred for tasks requiring nuanced understanding and language processing. These results offer valuable insights into the current state of digital information retrieval and pave the way for future innovations in this field. This study not only sheds light on the specific contexts in which each tool is favored but also hints at the potential for developing hybrid models that leverage the strengths of both search engines and LLMs. The insights gained from this research are pivotal for developers, researchers, and policymakers in understanding the evolving landscape of digital information retrieval and user interaction with these technologies.