# Benchmarking candidate coverage and rejection policy transfer in typed decision models

arXiv: https://arxiv.org/abs/2610.03387 · HTML: https://mlsanity.com/paper/2610.03387/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 9 of 9)

- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to It](https://mlsanity.com/paper/2609.26758/) (2026)
- ["None of the Above":Measure Uncertainty in Dialog Response Retrieval](https://mlsanity.com/paper/2004.01926/) (2020)
- [LEC: Linear Expectation Constraints for Selection-Conditioned Risk Control in Selective Prediction and Routing Systems](https://mlsanity.com/paper/2512.01556/) (2025)
- [Do System One Decisions Add Up? A Study of Probabilistic Coherence](https://mlsanity.com/paper/2609.33971/) (2026)
- [When No Answer Is Correct: Diagnosing Absent Answer Detection for MLLMs in Video Understanding](https://mlsanity.com/paper/2606.08239/) (2026)
- [When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jev](https://mlsanity.com/paper/2609.39496/) (2026)

## Built on by (most cited first, 1 of 1)

- [Lost in the Fallback: How Reliably Do Typed Decision Models Use Fallback Options?](https://mlsanity.com/paper/2610.19856/) (2026)

## Abstract (arXiv)

Rejection policies must remain useful as candidate sets and tasks change. We compare Laya, Jev and Qwen2.5-7B-Instruct using public reference labels, testing Laya/Jev policy transfer at equal calibration budgets and all three models on artificial omission, natural retrieval misses and public out-of-scope queries. Source calibration often fails to preserve the target operating point. A Jev policy calibrated on DBpedia rejects 69.3% of covered Emotion test inputs, while an Emotion policy loses detection entirely. Retrieval exposes a different tradeoff: with ten intent candidates, Laya detects 99.0% of out-of-scope queries but rejects 48.8% of covered queries. Separating missing-answer sources reveals these costs alongside retrieval coverage. The benchmark provides shared inputs, explicit decision and failure categories, and reproducible scoring to assess rejection policies under the conditions in which they are reused. Code and benchmark artifacts are available at https://github.com/luckykevvv/Decision_Model_Benchmark.