Exposure is not manifestation: measurement target and output resolution jointly determine which behavioural-faithfulness evaluator wins
Not on the map. Why?
Abstract (arXiv)
Behavioural auditing asks whether a language model behaves as it claims, but detection scores are reported without separating two targets: whether a reply was produced under a behaviour-inducing condition (exposure) and whether the behaviour surfaced in it (manifestation). Scoring a compact 146-million-parameter auditor's frozen-representation read-out and a frontier judge against each label on the identical 720 replies, the gap between the instruments moves by roughly 0.2 AUROC when the target changes. Under the judge's deployed interface, a single verdict, the ranking reverses: the auditor leads on exposure, 0.804 against 0.718, and trails on manifestation, 0.690 against 0.811. Matching the output resolution from either direction, by asking the judge a target-specific question answered with a continuous confidence score or by thresholding the auditor's read-out, removes the reversal but not the interaction, which excludes zero at all three resolutions (0.207, 0.237 and 0.169). The target governs how far apart the instruments are; the interface governs whether that distance changes their order. The auditor's hyperbolic geometry confers no advantage here. A single behavioural-detection AUROC is under-specified: such claims are comparable only when they state the estimand, the evaluator, and its output interface.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 83 of its 93 references that are in the map. First posted on arXiv 10 Jul 2026.
Builds on
83 of 93 references.
References · 83
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
- 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
- 2016Understanding intermediate layers using linear classifier probescited by 1,279
- 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
- 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
38 more
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
- 2018Model Cards for Model Reportingcited by 1,068
- 2020In Search of Lost Domain Generalizationcited by 1,042
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
- 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 674
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2023Textbooks Are All You Needcited by 561
- 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
- 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
- 2018Hyperbolic Neural Networkscited by 393
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
- 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
- 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
- 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
- 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
- 2023Language Models Represent Space and Timecited by 306
- 2019Measurement and Fairnesscited by 299
- 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
- 2018Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometrycited by 261
- 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 229
- 2018Representation Tradeoffs for Hyperbolic Embeddingscited by 204
- 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
- 2020Low-Dimensional Hyperbolic Knowledge Graph Embeddingscited by 195
- 2023Simple synthetic data reduces sycophancy in large language modelscited by 174
- 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 148
- 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 144
- 2021MultiVerS: Improving scientific claim verification with weak supervision and full-document contextcited by 140
- 2021Hyperbolic Deep Neural Networks: A Surveycited by 138
- 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 136
50 most cited of 83 All 83 (JSON)
Built on by
Not cited yet.