# Selection Is Retrieval, Abstention Is Not: On-Device Tool Routing over 70 Korean-English Actions

arXiv: https://arxiv.org/abs/2609.18672 · HTML: https://mlsanity.com/paper/2609.18672/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 9 of 9)

- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Multilingual E5 Text Embeddings: A Technical Report](https://mlsanity.com/paper/2402.05672/) (2024)
- [TinyAgent: Function Calling at the Edge](https://mlsanity.com/paper/2409.00608/) (2024)
- [Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents](https://mlsanity.com/paper/2603.29231/) (2026)
- [FunctionChat-Bench: Comprehensive Evaluation of Language Models' Generative Capabilities in Korean Tool-use Dialogs](https://mlsanity.com/paper/2411.14054/) (2024)
- [Benchmarking the Benchmarks: A Validity Audit of Tool-Calling Evaluation](https://mlsanity.com/paper/2607.02577/) (2026)
- [Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agents](https://mlsanity.com/paper/2605.08747/) (2026)
- [Most of the LLM Routing Gap Is Task Type](https://mlsanity.com/paper/2608.23023/) (2026)
- [SLM-Based Agentic AI with P-C-G: Optimized for Korean Tool Use](https://mlsanity.com/paper/2509.19369/) (2025)

## Abstract (arXiv)

An AI assistant that calls tools makes two decisions on every request: which tool to invoke, and whether any available tool applies. In the usual design a single language model makes both, by emitting a call or by declining to emit one. On a device that has to answer without a server, the language model is what makes that design expensive, dominating both the latency and the memory of the router. The common alternative is to remove the model completely and rank the catalog of local actions with a retriever instead. That substitution is not symmetric across the two decisions. A retriever returns its highest-scoring candidate for every input and cannot signal that the catalog holds no valid action. Our earlier study found that constraining a decoder to a tool grammar repairs malformed output without improving the choice. What the substitution costs in each decision has not been measured. We evaluate the two decisions separately over 600 Korean and English requests and a catalog of 70 local actions. The router may also ask for a missing slot, reply, or delegate. Half the in-catalog requests reuse catalog vocabulary and half paraphrase it, separating lexical overlap from the action requested. Character 3-gram BM25 selects 162 of 164 lexically matched requests and 85 of 166 paraphrases. Restricting the candidate set to seven raises the paraphrase figure to a mean of 0.825 over five trials. No classifier over its score features separates in-catalog from out-of-catalog above 0.697 area under the curve, where the frozen encoder multilingual-e5-base reaches 0.806. Using that encoder for abstention alone keeps 376 of the requests local and misroutes 9 of the 150 needing delegation. Abstention, not selection, is where a neural component is required. A neural ranker improves every quality metric and is rejected on latency and memory rather than accuracy.