# Auditing Proxy-Based Validation Across Text Spans

arXiv: https://arxiv.org/abs/2609.25808 · HTML: https://mlsanity.com/paper/2609.25808/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 94 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [RouterBench: A Benchmark for Multi-LLM Routing System](https://mlsanity.com/paper/2403.12031/) (2024)
- [SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models](https://mlsanity.com/paper/2311.08370/) (2023)
- [FreePRM: Training Process Reward Models Without Ground Truth Process Labels](https://mlsanity.com/paper/2506.03570/) (2025)
- [Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay](https://mlsanity.com/paper/2608.19760/) (2026)
- [Cross-Model Disagreement as a Label-Free Correctness Signal](https://mlsanity.com/paper/2603.25450/) (2026)
- [CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning](https://mlsanity.com/paper/2507.15698/) (2025)
- [What Does a Routing Oracle Measure Under Stochastic Decoding? Coupling, Scorer Choice, and Single-Commit Ceilings](https://mlsanity.com/paper/2607.03436/) (2026)
- [Building to the Test: Coding Agents Deliver What You Check, Not What You Requested](https://mlsanity.com/paper/2606.28430/) (2026)
- [A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation](https://mlsanity.com/paper/2608.24419/) (2026)
- [EST-PRM: Stress-Testing Process Reward Models Before They Become Load-Bearing](https://mlsanity.com/paper/2606.00437/) (2026)
- [Correct codes for the wrong reasons? validating LLMs as measurement instruments for theoretical constructs](https://mlsanity.com/paper/2606.28574/) (2026)
- [The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning](https://mlsanity.com/paper/2606.09078/) (2026)
- [When the Defense Writes the Refusal: Auditing Keyword-Scored Evaluation of Inference-Time Defenses for Multimodal Large Language Models](https://mlsanity.com/paper/2606.10904/) (2026)

## Abstract (arXiv)

Evaluation scores are often validated by their agreement with inexpensive proxy labels. When the score and the proxy are computed from the same text span, however, that agreement can arise from surface evidence the two share rather than from the semantic construct the proxy is meant to represent. We make the distinction explicit by declaring the score, its span, the proxy and the target construct as a validation contract, then re-evaluating that proxy rule strictly outside the scored span. In a controlled HotpotQA correctness experiment varying only the shared text boundary, the score agrees with its proxy far better than with correctness at a 50-character prefix: the gap is +0.184, collapsing to at most +0.045 from 120 characters onward. At that short prefix the score still predicts whether the answer string appears later (AUC 0.634) while an equivalence test places its agreement with correctness at chance, so the reported proxy agreement does not establish that the score ranks correctness. On OR-Bench, suppressing each model's recurring opening templates removes most of the score's association with the refusal proxy, while matched-volume deletion removes almost none and construct agreement stays at chance. Only three of eleven external contracts support the off-span control, and none of the routing studies we sampled released the generations it needs. We therefore ask that a proxy-based validation claim declare the span each label is read from, report the construct agreement beside the proxy agreement, and release the generations that let the proxy be re-read off the scored span.