mlsanityData to 14 Oct 2026

How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness

arXiv 2607.25068 ↗First posted 27 Jul 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Routing decisions between a cheap heuristic and an expensive large language model (LLM) are typically framed as a difficulty problem: send the hard cases to the expensive path. We argue this framing is incomplete because difficulty and business value are distinct axes - a difficult cheap item and a difficult costly item do not have the same cost of error. We present Value Router, a fully synthetic simulation of a retail merchandising pipeline that routes items using only estimated difficulty and estimated value, never ground truth. The study has three stages. First, a value-weighted threshold router is compared with a difficulty-only and a random baseline on a synthetic catalog with an inverse correlation between category volume and value. Value-weighting matches the difficulty-only baseline's recall of true high-value items (60%) while achieving substantially higher precision (98.3% vs. 94.3%). Second, a decision logger and monitor expose a failure mode hidden by aggregate metrics showing that the aggregate result is driven almost entirely by between-category differences rather than per-item discrimination. Third, a simulated Black Friday demand surge (2.5 volume with a shift toward higher-value categories) compares a static router, a seasonally tuned router, and two slow-path budget policies. All results are from a controlled synthetic simulation with experimenter-defined ground truth and illustrate design principles for cost-aware routing systems rather than validated real-world claims.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 17 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

9 of 17 references.

References · 9

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  5. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  6. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  7. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  8. 2022Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systemscited by 44
  9. 2020FrugalML: How to Use ML Prediction APIs More Accurately and Cheaplycited by 33

Built on by

Not cited yet.