Auditable Decision Models with Learned Abstention and Real-Time Steering
Not on the map. Why?
Abstract (arXiv)
Production AI systems often operate with incomplete, conflicting, or insufficient evidence. Forced classifiers collapse such cases into action labels, while generative systems can produce outputs that are difficult to interpret as auditable execution decisions. We study operational decision control for AI systems, where uncertainty must be explicitly routable, policy-governed, and auditable rather than hidden inside forced predictions or free-form generation. We present EvaluatorDPT, a bounded decision-control model that predicts YES, NO, or TBD, where TBD is learned as a deferral outcome rather than added only as a post-hoc confidence rule. The model uses a transformer encoder with a primary bounded-decision head and structured auxiliary channels for values and emotions/sentiments. The interface is domain-agnostic in form: a deployment domain supplies evidence and policy thresholds, while the model emits a bounded distribution that can be controlled at inference time through recorded operating thresholds and, when validated, auxiliary semantic signals. For the evaluated model version, we report decision performance on held-out validation and test splits; auxiliary emotion metrics are omitted because the emotion head is disabled for this evaluation. On the held-out test split (n=44,597), the model achieves Accuracy = 0.8260 and Macro F1 = 0.8252, with per-class F1 of 0.8314 (YES), 0.8486 (NO), and 0.7956 (TBD). The evaluation record also includes calibration evidence (ECE = 0.0338 on validation), threshold-sweep outputs, multi-seed stability checks, confusion matrices, and reproducibility commands. Our main contribution is a bounded execution interface in which deferral is learned, inference-time routing remains inspectable, auxiliary signals provide a path to auditable behavior control, and evaluation evidence supports external review.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 26 May 2026.
Builds on
14 of 24 references.
References · 14
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
- 2015A large annotated corpus for learning natural language inferencecited by 1,837
- 2017Selective Classification for Deep Neural Networkscited by 799
- 2020Aligning AI With Shared Human Valuescited by 706
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
- 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
- 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
- 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
- 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
- 2019SemEval-2017 Task 4: Sentiment Analysis in Twittercited by 209
2 more
- 2021Optimal strategies for reject option classifierscited by 61
- 2023NormBank: A Knowledge Bank of Situational Social Normscited by 60
Built on by
Not cited yet.