mlsanityData to 14 Oct 2026

Labels Override Definitions in Jev-Style Typed Decision Models

arXiv 2610.02586 ↗First posted 1 Oct 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

A typed decision model answers a fixed question about an input by returning a probability for each of several caller-defined options. Each option carries a short label and a written definition, which is where a developer states the rule the model should apply. Jev introduced this interface for routing, moderation and triage, open implementations followed, and the same operation occurs whenever a language model is used as a classifier by scoring label strings. We study the open implementations, whose weights we can inspect and patch, and ask whether the probability follows the definitions or the labels. A preference for the label we call option-label bias. Across four open-weight typed decision models, three ways of reading an answer from a Qwen2.5 backbone, eleven classification tasks and PolicyBench, a synthetic routing suite we introduce in which the rule appears only in the definitions, the answer is mostly the labels. Deleting every definition leaves accuracy unchanged (laya-td: 0.8559 against 0.8487), although those definitions support 0.7971 on their own, and renaming the options to A and B raises accuracy by +0.1511 [+0.1377, +0.1646]. One system, von, is unaffected, and the two code bases differ in one expression: laya writes each option as "{label}: {definition}", while von writes only the definition. Changing that expression in both directions, with no weight changed, makes all three laya checkpoints exactly invariant (+0.0000 [+0.0000, +0.0000]) and creates the effect in von, whose accuracy falls from 0.8511 to 0.2281 when a label contradicts its definition. Earlier work attributed this failure to the constrained decision head these models use in place of a text decoder; our results locate it in the prompt rendering. We give a two-call test that tells a practitioner which case applies to their model, and measure what four mitigations are worth.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

12 of 16 references.

References · 12

  1. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  2. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  3. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  4. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
  5. 2026JEV-as-a-Judge: Accept When Confident, Escalate When Unsurecited by 30
  6. 2026Type-Safe Is Not Error-Free: Typed Decision Models Follow the Option Name, Not the Definition Bound to Itcited by 25
  7. 2026Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisionscited by 10
  8. 2026JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Modelscited by 10
  9. 2026Evaluating and Benchmarking the System One Model Jevcited by 9
  10. 2026Jev in the Wild: A Data-Driven Analysis of the Jev Model's Functionality, Applications and Ecosystemcited by 7
  11. 2026Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?cited by 6
  12. 2026Typed Decision Models: An Early Evidence Audit and Evaluation Checklistcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.