The Stability Trap: Evaluating the Reliability of LLM-Based Instruction Adherence Auditing
Not on the map. Why?
Abstract (arXiv)
The enterprise governance of Generative AI (GenAI) in regulated sectors, such as Human Resources (HR), demands scalable yet reproducible auditing mechanisms. While Large Language Model (LLM)-as-a-Judge approaches offer scalability, their reliability in evaluating adherence of different types of system instructions remains unverified. This study asks: To what extent does the instruction type of an Application Under Test (AUT) influence the stability of judge evaluations? To address this, we introduce the Scoped Instruction Decomposition Framework to classify AUT instructions into Objective and Subjective types, isolating the factors that drive judge instability. We applied this framework to two representative HR GenAI applications, evaluating the stability of four judge architectures over variable runs. Our results reveal a ``Stability Trap'' characterized by a divergence between Verdict Stability and Reasoning Stability. While judges achieved near-perfect verdict agreement (>99%) for both objective and subjective evaluations, their accompanying justification traces diverged significantly. Objective instructions requiring quantitative analysis, such as word counting, exhibited reasoning stability as low as ≈19%, driven by variances in numeric justifications. Similarly, reasoning stability for subjective instructions varied widely (35%--83%) based on evidence granularity, with feature-specific checks failing to reproduce consistent rationale. Conversely, objective instructions focusing on discrete entity extraction achieved high reasoning stability (>90%). These findings demonstrate that high verdict stability can mask fragile reasoning. Thus, we suggest that auditors scope automated evaluation protocols strictly: delegate all deterministically verifiable logic to code, while reserving LLM judges for complex semantic evaluation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 35 of its 76 references that are in the map. First posted on arXiv 16 Jan 2026.
Builds on
35 of 76 references.
References · 35
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
- 2023A Survey on Evaluation of Large Language Modelscited by 1,242
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
- 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
23 more
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
- 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
- 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
- 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 302
- 2024TrustLLM: Trustworthiness in Large Language Modelscited by 285
- 2023FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Setscited by 201
- 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
- 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 117
- 2024Non-Determinism of "Deterministic" LLM Settingscited by 92
- 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 87
- 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 38
- 2025LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judgecited by 37
- 2021Conformity Assessments and Post-market Monitoring: A Guide to the Role of Auditing in the Proposed European AI Regulationcited by 19
- 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMscited by 17
- 2024AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approachcited by 11
- 2025When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validitycited by 11
- 2022A Human-Centric Take on Model Monitoringcited by 9
- 2025SpecEval: Evaluating Model Adherence to Behavior Specificationscited by 9
- 2024A Framework for Assurance Audits of Algorithmic Systemscited by 8
- 2025Challenges in Testing Large Language Model Based Software: A Faceted Taxonomycited by 8
- 2025When Models Can't Follow: Testing Instruction Adherence Across 256 LLMscited by 5
- 2024A Blueprint for Auditing Generative AIcited by 4
- 2025IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluationcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.