Preference Is Not Intervention: The Structure and Stability Boundaries of Reader-Specific Evidence Utility
Not on the map. Why?
Abstract (arXiv)
ML systems increasingly condition decisions on downstream model identity, but this is useful only if model-specific differences form reusable structure rather than input-local interactions. We test this in retrieval-augmented generation (RAG), where evidence utility can be measured under controlled interventions. Holding query, evidence, task, scoring, and intervention fixed, nine readers disagree on effect sign in 33% of jointly affected cells; reader×query interaction explains 29.8% of utility variance versus an 8.4% permutation null; and self-selected evidence improves F1 by +0.031 (t=3.39). We then ask the sharper question: which components of this heterogeneity are stable reader properties across queries? Separating three measurable objects---evidence activity, ordinal preference, and conditional signed direction---we find ordinal reader geometry stable across four independent settings (split-half ρ=0.60--0.83): leave-one-out interventions, PRISM preferences, RAMDocs, and RAGuard. Signed geometry is task-bounded: weak in open-ended QA (0.14, 0.35), especially for misleading and irrelevant evidence, but strong in binary fact-checking (0.75) with no significant ordinal gap, though still below its sparsity-matched ceiling. Sparsity, decoding noise, and metric artifacts do not explain the main ordinal--signed gap. Finally, stable ordinal similarity fails to predict cross-reader intervention transfer (oracle-distance ρ=-0.27; regret reliability -0.28). Reader-specific utility exists, but preference is not intervention: stable ranking similarity does not license transfer of help/harm decisions.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 14 references that are in the map. First posted on arXiv 18 Aug 2026.
Builds on
12 of 14 references.
References · 12
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
- 2024Bridging the Preference Gap between Retrievers and LLMscited by 69
- 2025Retrieval-Augmented Generation with Conflicting Evidencecited by 63
- 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievalscited by 28
- 2024Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Modelscited by 23
- 2024Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximizationcited by 14
- 2025MAGIC: A Multi-Hop and Graph-Based Benchmark for Inter-Context Conflicts in Retrieval-Augmented Generationcited by 6
- 2025LLM-Specific Utility for Retrieval-Augmented Generationcited by 4
- 2026Rank4Gen: RAG-Preference-Aligned Document Set Selection and Rankingcited by 4
- 2026LURE-RAG: Lightweight Utility-driven Reranking for Efficient RAGcited by 2
- 2026Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026CUE-R: Beyond the Final Answer in Retrieval-Augmented Generation
- 2026Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMs
- 2026The RAT: A Unified Bayesian Model for RAG Evaluation
- 2026Can Revealed Preferences Clarify LLM Alignment and Steering?
- 2026Assessing the Downstream Utility of Evidence-Aware Retrieval in RAG
- 2026When More Evidence Hurts: Publication-Bias Drift and Principled Stopping for Biomedical Causal Search
- 2026Utility-Oriented Visual Evidence Selection for Multimodal Retrieval-Augmented Generation
- 2026Search Shapes Conclusions: Auditing Evidence Selection Bias in Deep Research Agents