When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jev
Not on the map. Why?
Abstract (arXiv)
Typed decision models such as Jev offer an efficient alternative to generative LLMs in decision-making workflows by selecting directly from predefined options. When candidate sets contain no valid answer, TypeSafe recommends including an "other" or "none-of-the-above" option to enable rejection. In this report, however, we identify an arithmetic-dependent rejection bottleneck: Jev reliably selects correct numerical answers when available but frequently accepts incorrect alternatives when they are absent despite an explicit rejection option. On paired arithmetic problems, answer-present accuracy reaches 99%, while correct rejection falls to 7%. Moreover, this gap persists across numerical magnitudes, operation depths, contextual formulations, and rejection labels, and extends to scenarios such as time calculation and capacity rounding. Yet native Boolean verification achieves 99% exact-match accuracy on the same answer-absent arithmetic cases, showing that categorical rejection can fail even when the model successfully verifies candidate correctness. Finally, we show that a simple decision threshold selected on separate development problems raises arithmetic rejection accuracy from 7% to 79% while retaining 97% answer-present accuracy, substantially mitigating the failure without retraining or additional inference.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 6 of its 12 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
6 of 12 references.
References · 6
- 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
- 2024Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Modelscited by 57
- 2024LLMs May Perform MCQA by Selecting the Least Incorrect Optioncited by 33
- 2026Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to Itcited by 25
- 2026Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understandingcited by 11
- 2025None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answeringcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.