# Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences

arXiv: https://arxiv.org/abs/2609.24965 · HTML: https://mlsanity.com/paper/2609.24965/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 38 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Builds on (most cited first, 3 of 3)

- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers](https://mlsanity.com/paper/2506.10486/) (2025)

## Built on by (most cited first, 12 of 12)

- [Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?](https://mlsanity.com/paper/2609.33209/) (2026)
- [Calibrated Decision Models for Autonomous Penetration-Testing Harnesses: JEV and Laya as System One Decision Layers for LLM-Driven Pentest Agents](https://mlsanity.com/paper/2609.28940/) (2026)
- [Typed Decision Models: An Early Evidence Audit and Evaluation Checklist](https://mlsanity.com/paper/2609.32160/) (2026)
- [Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces](https://mlsanity.com/paper/2609.37470/) (2026)
- [From Text Decisions to Pixels: An Study of Jev-Style Visual Choice Model](https://mlsanity.com/paper/2609.29283/) (2026)
- [Benchmarking System One decision models against trained classifiers and language models for automated decision gates](https://mlsanity.com/paper/2610.00346/) (2026)
- [Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite](https://mlsanity.com/paper/2610.02826/) (2026)
- [Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use](https://mlsanity.com/paper/2610.04985/) (2026)
- [Beyond Answer Confidence: A Controlled Audit of Self-Knowledge in a Black-Box Decision Model](https://mlsanity.com/paper/2610.01006/) (2026)
- [Can Jev be Your Q or Policy in Reinforcement Learning?](https://mlsanity.com/paper/2610.11692/) (2026)
- [Can Decision Models Understand Stance? Evaluating Jev Against General-Purpose LLMs](https://mlsanity.com/paper/2610.11901/) (2026)
- [Specialized Decision Models vs. General-Purpose LLMs: Benchmarking Jev Across Knowledge, Reasoning, and Multilingual Tasks](https://mlsanity.com/paper/2610.11978/) (2026)

## Abstract (arXiv)

Scientific workflows often require choosing among known relations before a deterministic calculation can proceed. Whether observations share a culture, treatment or reference standard can change the scientific meaning of the resulting count or comparison. We evaluate Jev as a semantic decision component using a harness that follows its documented guidance and assigns arithmetic to code. The study compares twelve model configurations on twenty source-grounded Choices across ten scientific cases, each repeated five times. We measure semantic selections, downstream outputs and final claim labels separately. Jev matched five other configurations at complete semantic correctness and achieved the lowest observed median latency among successful responses. Across three comparison models, seven wrong selections on one culture-history question changed downstream counts while preserving the correct final label. These results identify a useful role for Jev in prepared scientific decision tasks and show why evaluating that role requires checking the relations and quantities that a workflow will reuse.