mlsanityData to 7 Oct 2026

Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences

arXiv 2609.24965 ↗First posted 21 Sep 2026cs.CL cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Scientific workflows often require choosing among known relations before a deterministic calculation can proceed. Whether observations share a culture, treatment or reference standard can change the scientific meaning of the resulting count or comparison. We evaluate Jev as a semantic decision component using a harness that follows its documented guidance and assigns arithmetic to code. The study compares twelve model configurations on twenty source-grounded Choices across ten scientific cases, each repeated five times. We measure semantic selections, downstream outputs and final claim labels separately. Jev matched five other configurations at complete semantic correctness and achieved the lowest observed median latency among successful responses. Across three comparison models, seven wrong selections on one culture-history question changed downstream counts while preserving the correct final label. These results identify a useful role for Jev in prepared scientific decision tasks and show why evaluating that role requires checking the relations and quantities that a workflow will reuse.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 3 of its 38 references that are in the map. First posted on arXiv 21 Sep 2026.

Builds on

3 of 38 references.

References · 3

  1. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  2. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  3. 2025Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Paperscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.