# Preference Is Not Intervention: The Structure and Stability Boundaries of Reader-Specific Evidence Utility

arXiv: https://arxiv.org/abs/2608.17781 · HTML: https://mlsanity.com/paper/2608.17781/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 18 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Bridging the Preference Gap between Retrievers and LLMs](https://mlsanity.com/paper/2401.06954/) (2024)
- [Retrieval-Augmented Generation with Conflicting Evidence](https://mlsanity.com/paper/2504.13079/) (2025)
- [Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals](https://mlsanity.com/paper/2502.16101/) (2025)
- [Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.00175/) (2024)
- [Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization](https://mlsanity.com/paper/2410.09942/) (2024)
- [MAGIC: A Multi-Hop and Graph-Based Benchmark for Inter-Context Conflicts in Retrieval-Augmented Generation](https://mlsanity.com/paper/2507.21544/) (2025)
- [LLM-Specific Utility for Retrieval-Augmented Generation](https://mlsanity.com/paper/2510.11358/) (2025)
- [Rank4Gen: RAG-Preference-Aligned Document Set Selection and Ranking](https://mlsanity.com/paper/2601.11273/) (2026)
- [LURE-RAG: Lightweight Utility-driven Reranking for Efficient RAG](https://mlsanity.com/paper/2601.19535/) (2026)
- [Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning](https://mlsanity.com/paper/2604.02091/) (2026)

## Abstract (arXiv)

ML systems increasingly condition decisions on downstream model identity, but this is useful only if model-specific differences form reusable structure rather than input-local interactions. We test this in retrieval-augmented generation (RAG), where evidence utility can be measured under controlled interventions. Holding query, evidence, task, scoring, and intervention fixed, nine readers disagree on effect sign in 33% of jointly affected cells; reader×query interaction explains 29.8% of utility variance versus an 8.4% permutation null; and self-selected evidence improves F1 by +0.031 (t=3.39). We then ask the sharper question: which components of this heterogeneity are stable reader properties across queries? Separating three measurable objects---evidence activity, ordinal preference, and conditional signed direction---we find ordinal reader geometry stable across four independent settings (split-half ρ=0.60--0.83): leave-one-out interventions, PRISM preferences, RAMDocs, and RAGuard. Signed geometry is task-bounded: weak in open-ended QA (0.14, 0.35), especially for misleading and irrelevant evidence, but strong in binary fact-checking (0.75) with no significant ordinal gap, though still below its sparsity-matched ceiling. Sparsity, decoding noise, and metric artifacts do not explain the main ordinal--signed gap. Finally, stable ordinal similarity fails to predict cross-reader intervention transfer (oracle-distance ρ=-0.27; regret reliability -0.28). Reader-specific utility exists, but preference is not intervention: stable ranking similarity does not license transfer of help/harm decisions.