# Learning the Cost of Reliable Inference

arXiv: https://arxiv.org/abs/2609.28322 · HTML: https://mlsanity.com/paper/2609.28322/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 53 references that are in the map.
- First posted on arXiv 23 Sep 2026.

## Builds on (most cited first, 8 of 8)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts](https://mlsanity.com/paper/2406.12845/) (2024)
- [Competition and Diversity in Generative AI](https://mlsanity.com/paper/2412.08610/) (2024)
- [Economic Evaluation of LLMs](https://mlsanity.com/paper/2507.03834/) (2025)
- [Test-Time Compute Games](https://mlsanity.com/paper/2601.21839/) (2026)
- [The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More](https://mlsanity.com/paper/2603.23971/) (2026)
- [An Incentive Compatible Multi-Armed-Bandit Crowdsourcing Mechanism with Quality Assurance](https://mlsanity.com/paper/1406.7157/) (2014)
- [Contextual Procurement Auctions with Bandit Learning](https://mlsanity.com/paper/2607.05813/) (2026)

## Abstract (arXiv)

Benchmarking and routing platforms increasingly act as intermediaries connecting large language model providers with end-users. However, providers on these platforms typically use a fixed price per token, preventing users from achieving the most competitive price for their tasks. In this work, we design a procurement platform where token prices for each task are driven by provider competition, enabling users to secure competitive pricing for guaranteed quality levels. To this end, the platform sequentially routes queries via a reverse second-price auction that incentivizes model providers to truthfully bid their best estimate of the average cost to serve a user's query. As it routes queries, the platform learns the quality offered by each provider and progressively routes queries to the most cost-competitive provider among those meeting a desired quality threshold. To validate our design, we conduct experiments with multiple LLMs from the Llama and Qwen families on popular mathematical reasoning and question-answering benchmarks. The results show that the pricing margin of the most cost-competitive provider on our platform varies significantly---from 10% to 71%---depending on the task and quality threshold. This suggests a substantial inefficiency in the current fixed-price market, and it demonstrates that our platform may enable users to capture maximum savings whenever competitive market conditions permit.