mlsanityData to 14 Oct 2026

When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jev

arXiv 2609.39496 ↗First posted 30 Sep 2026cs.LG cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Typed decision models such as Jev offer an efficient alternative to generative LLMs in decision-making workflows by selecting directly from predefined options. When candidate sets contain no valid answer, TypeSafe recommends including an "other" or "none-of-the-above" option to enable rejection. In this report, however, we identify an arithmetic-dependent rejection bottleneck: Jev reliably selects correct numerical answers when available but frequently accepts incorrect alternatives when they are absent despite an explicit rejection option. On paired arithmetic problems, answer-present accuracy reaches 99%, while correct rejection falls to 7%. Moreover, this gap persists across numerical magnitudes, operation depths, contextual formulations, and rejection labels, and extends to scenarios such as time calculation and capacity rounding. Yet native Boolean verification achieves 99% exact-match accuracy on the same answer-absent arithmetic cases, showing that categorical rejection can fail even when the model successfully verifies candidate correctness. Finally, we show that a simple decision threshold selected on separate development problems raises arithmetic rejection accuracy from 7% to 79% while retaining 97% answer-present accuracy, substantially mitigating the failure without retraining or additional inference.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 6 of its 12 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

6 of 12 references.

References · 6

  1. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  2. 2024Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Modelscited by 57
  3. 2024LLMs May Perform MCQA by Selecting the Least Incorrect Optioncited by 33
  4. 2026Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to Itcited by 25
  5. 2026Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understandingcited by 11
  6. 2025None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answeringcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.