Curated retrieval versus open web search in public AI information services: a coverage-trust trade-off
Not on the map. Why?
Abstract (arXiv)
Public institutions increasingly use large language models (LLMs) to answer citizens' questions, often pairing a curated knowledge base with live web search, yet whether the sources behind these answers can be trusted has received little empirical scrutiny. We report a pre-launch expert evaluation of Evr'opuvefur, an independent, government-funded service run by the University of Iceland that answers questions about the European Union, conducted as Iceland prepared for its referendum of 29 August 2026 on whether to resume EU accession talks. Five domain experts produced 551 evaluations of 449 AI-generated answers, scoring each against a seven-criterion quality rubric and, separately, flagging individual cited sources. We compared two retrieval paths: a curated local corpus (RAG) and open web search. In more than a third of the reviewed web-search answers (35%, 65 of 187), at least one cited source was flagged, almost always as untrustworthy or irrelevant; curated sources were flagged far less often and only for being out of date. Web search answered more questions, but at the cost of source quality; the curated corpus was trustworthy yet limited in coverage, and the model declined to respond when it fell short. The citation mix also passed over strong sources: across all 287 web-search answers, the system never cited R'UV, the public broadcaster and the country's most widely used news source. A companion prompt ablation shows how weak prompt-level steering is: a trusted-domain list in the system prompt raised the share of citations to listed domains only from 12% to 21%. Fluency and topical fit did not predict source trustworthiness. We argue that source trustworthiness is a measurable yet largely invisible dimension of information quality in public AI services, and we discuss transparency-oriented responses and their trade-offs.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
22 of 33 references.
References · 22
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 302
- 2020Information-Theoretic Probing for Linguistic Structurecited by 204
- 2018SpaceNet: A Remote Sensing Dataset and Challenge Seriescited by 178
- 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 167
- 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 145
- 2021Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networkscited by 141
- 2024Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applicationscited by 139
10 more
- 2022SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observationcited by 82
- 2022DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentationcited by 78
- 2021SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learningcited by 53
- 2021Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Seriescited by 49
- 2022Enabling Country-Scale Land Cover Mapping with Meter-Resolution Satellite Imagerycited by 31
- 2024A Closer Look at Benchmarking Self-Supervised Pre-training with Image Classificationcited by 19
- 2025Measuring the Intrinsic Dimension of Earth Representationscited by 10
- 2026EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observationcited by 8
- 2025Towards a Unified Representation Evaluation Framework Beyond Downstream Taskscited by 7
- 2025NeuCo-Bench: A Novel Benchmark Framework for Neural Embeddings in Earth Observationcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Navigating Global AI Regulation: A Multi-Jurisdictional Retrieval-Augmented Generation System
- 2026Similar Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents
- 2026TRACE: Accountable Agentic Retrieval for Source Discovery in Digital Archives
- 2026Retrieval Improvements Do Not Guarantee Better Answers: A Study of RAG for AI Policy QA
- 2025PolicyBot - Reliable Question Answering over Policy Documents
- 2026SimpleWikiSearch: A Clean Offline Wikipedia Environment for Agentic Search
- 2026CIVI: A Framework for Diagnosing Search Agent Failures in Civic Information
- 2025Safety Degradation in AI Agents