# Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration

arXiv: https://arxiv.org/abs/2609.22753 · HTML: https://mlsanity.com/paper/2609.22753/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2026.

## Built on by (most cited first, 9 of 9)

- [Typed Decision Models: An Early Evidence Audit and Evaluation Checklist](https://mlsanity.com/paper/2609.32160/) (2026)
- [Benchmarking System One decision models against trained classifiers and language models for automated decision gates](https://mlsanity.com/paper/2610.00346/) (2026)
- [Jev Matches 7B Language Models for Speech-Neuroprosthesis Rescoring](https://mlsanity.com/paper/2609.33538/) (2026)
- [Bongard: Training Machine Intuition](https://mlsanity.com/paper/2609.39111/) (2026)
- [OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System One](https://mlsanity.com/paper/2610.00381/) (2026)
- [SoK: Semantic Decision Engines in Network Control Loops](https://mlsanity.com/paper/2610.06425/) (2026)
- [Readout Stability in Prefill-Only Decision Models:Zero-Label Prediction and Inference-Time Compute Allocation](https://mlsanity.com/paper/2610.07716/) (2026)
- [Token-Efficient Multi-Agent Collaboration via System One-Guided Computational Division of Labor](https://mlsanity.com/paper/2610.08155/) (2026)
- [Can Jev be Your Q or Policy in Reinforcement Learning?](https://mlsanity.com/paper/2610.11692/) (2026)

## Abstract (arXiv)

Natural-language service requests can require a language-model decision before execution starts, consuming part of the request's latency budget. We integrate Jev's decision-oriented application programming interface (API) into edge service orchestration to reduce this overhead while retaining service completion. The integration extracts four bounded intent fields and applies a shared validator, admission policy, and scheduler, accounting for decision waiting throughout the request timeline. We compare Jev with a short, structured-output DeepSeek deployment using live API measurements followed by modeled execution, and then a real two-node optical character recognition (OCR) service with self-hosted Qwen and rule-based references. Across three consecutive measurement blocks, Jev reduces median client decision latency by 15.9-26.5%. In eight paired OCR conditions, Jev matches DeepSeek's correct, on-time completion count in seven and exceeds it in one. Without caching, median end-to-end latency on requests completed correctly by both systems is 11.1-25.3% lower; API fees per correct completion are 69.0-70.6% lower. Repeated-request caching largely removes the latency difference. The results demonstrate that decision-model substitution can lower both response latency and API fees in the tested service path, and identify fresh interpretation as the main opportunity for latency savings.