How Often Should a Recommender Call an LLM? Value-Weighted Routing, Monitoring, and Seasonal Robustness
Not on the map. Why?
Abstract (arXiv)
Routing decisions between a cheap heuristic and an expensive large language model (LLM) are typically framed as a difficulty problem: send the hard cases to the expensive path. We argue this framing is incomplete because difficulty and business value are distinct axes - a difficult cheap item and a difficult costly item do not have the same cost of error. We present Value Router, a fully synthetic simulation of a retail merchandising pipeline that routes items using only estimated difficulty and estimated value, never ground truth. The study has three stages. First, a value-weighted threshold router is compared with a difficulty-only and a random baseline on a synthetic catalog with an inverse correlation between category volume and value. Value-weighting matches the difficulty-only baseline's recall of true high-value items (60%) while achieving substantially higher precision (98.3% vs. 94.3%). Second, a decision logger and monitor expose a failure mode hidden by aggregate metrics showing that the aggregate result is driven almost entirely by between-category differences rather than per-item discrimination. Third, a simulated Black Friday demand surge (2.5 volume with a shift toward higher-value categories) compares a static router, a seasonally tuned router, and two slow-path budget policies. All results are from a controlled synthetic simulation with experimenter-defined ground truth and illustrate design principles for cost-aware routing systems rather than validated real-world claims.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 17 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
9 of 17 references.
References · 9
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
- 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
- 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
- 2022Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systemscited by 44
- 2020FrugalML: How to Use ML Prediction APIs More Accurately and Cheaplycited by 33
Built on by
Not cited yet.