Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to It
Not on the map. Why?
Abstract (arXiv)
Typed decision models return structured results, but output-type correctness alone does not ensure that decisions follow explicit option definitions. Each option pairs a name with a definition that defines its intended meaning; the name, however, can provide a competing semantic cue. We study this conflict in Jev and two open-weight models by changing only the name-definition mapping, leaving the question, state, and the names and definition texts themselves unchanged. We measure decision flips at the level of the selected definition, rather than the returned name. On 1200 decision tasks with task-specific definitions, decision-flip rates are up to 70.4 pp higher with yes/no names than with the 0/1 control. This gap holds across all 4 binary decision rules. With yes/no names, reassignment also lowers their mean AUC from 93.8% to a below-chance 23.2%. In the binary evaluations, random strings used as option names yield mean flip rates close to those of neutral controls across all three models, with comparable balanced accuracy before reassignment. Together, these results support option-name polarity as a contributor to decision instability beyond reassignment alone. The type-error rate remains 0% throughout, showing that type-correct outputs can still fail to follow explicit option definitions.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 22 Sep 2026.
Builds on
11 of 20 references.
References · 11
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
- 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
- 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
- 2023Larger language models do in-context learning differentlycited by 386
- 2023Mitigating Word Bias in Zero-shot Prompt-based Classifierscited by 9
- 2026The Structured Output Benchmark: A Multi-Source Benchmark for Evaluating Structured Output Quality in Large Language Modelscited by 3
- 2026Schema-Key Wording as an Instruction Channel in Structured Generation under Constrained Decodingcited by 2
- 2026Your Prompt Is Not the Only Prompt: How Much Do LLMs Weight Structured-Output Schema Descriptions?cited by 2
- 2025Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labelscited by 1
- 2026When JSON Is Not Enough: Semantic Reliability of Schema-Constrained LLM Ordering Agentscited by 1
- 2026PhantomFill: When the Form Demands an Answer, Language Models Invent Onecited by 1
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026JevOut: Natural Context Can Flip Decision Modelscited by 4
- 2026JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Modelscited by 1
- 2026Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?cited by 1
- 2026Typed Decision Models: An Early Evidence Audit and Evaluation Checklistcited by 0
- 2026Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automationcited by 0
- 2026Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfacescited by 0
- 2026When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jevcited by 0
- 2026Benchmarking System One decision models against trained classifiers and language models for automated decision gatescited by 0
- 2026OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System Onecited by 0
- 2026AnyJev Technical Reportcited by 0
- 2026Adversarial Cues in Decision Models Used as Judges: The Role of Request Presentationcited by 0
- 2026Can Jev be Your Q or Policy in Reinforcement Learning?cited by 0
- 2026Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks8 Oct 2026
- 2026Can Jev be Your Q or Policy in Reinforcement Learning?8 Oct 2026
- 2026Adversarial Cues in Decision Models Used as Judges: The Role of Request Presentation8 Oct 2026
- 2026AnyJev Technical Report30 Sep 2026
- 2026OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One30 Sep 2026
- 2026When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jev30 Sep 2026
- 2026Benchmarking System One decision models against trained classifiers and language models for automated decision gates29 Sep 2026
- 2026Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces27 Sep 2026
- 2026Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation27 Sep 2026
- 2026Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?27 Sep 2026
- 2026Typed Decision Models: An Early Evidence Audit and Evaluation Checklist26 Sep 2026
- 2026JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models25 Sep 2026
1 more
- 2026JevOut: Natural Context Can Flip Decision Models24 Sep 2026
- 2026JevOut: Natural Context Can Flip Decision Modelscited by 4
- 2026JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Modelscited by 1
- 2026Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?cited by 1
- 2026Typed Decision Models: An Early Evidence Audit and Evaluation Checklistcited by 0
- 2026Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfacescited by 0
- 2026When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jevcited by 0
- 2026Benchmarking System One decision models against trained classifiers and language models for automated decision gatescited by 0
- 2026OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System Onecited by 0
- 2026AnyJev Technical Reportcited by 0
- 2026Adversarial Cues in Decision Models Used as Judges: The Role of Request Presentationcited by 0
- 2026Can Jev be Your Q or Policy in Reinforcement Learning?cited by 0
- 2026Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Taskscited by 0
- 2026Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks8 Oct 2026
- 2026Can Jev be Your Q or Policy in Reinforcement Learning?8 Oct 2026
- 2026Adversarial Cues in Decision Models Used as Judges: The Role of Request Presentation8 Oct 2026
- 2026AnyJev Technical Report30 Sep 2026
- 2026OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One30 Sep 2026
- 2026When the Right Answer Is Missing: An Arithmetic-Dependent Rejection Bottleneck in Jev30 Sep 2026
- 2026Benchmarking System One decision models against trained classifiers and language models for automated decision gates29 Sep 2026
- 2026Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces27 Sep 2026
- 2026Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?27 Sep 2026
- 2026Typed Decision Models: An Early Evidence Audit and Evaluation Checklist26 Sep 2026
- 2026JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models25 Sep 2026
- 2026JevOut: Natural Context Can Flip Decision Models24 Sep 2026