# Don't Offer What Can't Be Done: Deterministic Executability Gating for LLM Skill Selection at Scale

arXiv: https://arxiv.org/abs/2608.01050 · HTML: https://mlsanity.com/paper/2608.01050/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent skill learning and evolution, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 2 Aug 2026.

## Placed in

- LLM agents and tool use / [Agent skill learning and evolution](https://mlsanity.com/agents/agent-skill-learning-and-evolution/)

## Builds on (most cited first, 16 of 16)

- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities](https://mlsanity.com/paper/2408.04682/) (2024)
- [Toolshed: Scale Tool-Equipped Agents with Advanced RAG-Tool Fusion and Tool Knowledge Bases](https://mlsanity.com/paper/2410.14594/) (2024)
- [LongFuncEval: Measuring the effectiveness of long context models for function calling](https://mlsanity.com/paper/2505.10570/) (2025)
- [ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agents](https://mlsanity.com/paper/2606.06284/) (2026)
- [When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents](https://mlsanity.com/paper/2602.08995/) (2026)
- [Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use](https://mlsanity.com/paper/2603.03205/) (2026)
- [ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents](https://mlsanity.com/paper/2606.15508/) (2026)
- [RAGXplain: From Explainable Evaluation to Actionable Guidance of RAG Pipelines](https://mlsanity.com/paper/2505.13538/) (2025)
- [PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models](https://mlsanity.com/paper/2607.05441/) (2026)
- [Building Customer Support AI Agents at 100M-User Scale: An Evaluation-Driven Framework](https://mlsanity.com/paper/2606.08867/) (2026)
- [GIST-CMTF: Goal-State Inference for Causal Minimal Tool Filtering in LLM Agents](https://mlsanity.com/paper/2606.16813/) (2026)
- [Agent Security Needs Redefinition through a Holistic Framework](https://mlsanity.com/paper/2607.22024/) (2026)

## Built on by (most cited first, 1 of 1)

- [What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents](https://mlsanity.com/paper/2609.39989/) (2026)

## Abstract (arXiv)

Production LLM agents that select from large skill libraries face a limitation that semantic relevance alone cannot resolve: a skill may match a user's topic yet be impossible to execute in the current account state. We present a deployed three-stage selection pipeline for Helpmate, Wix's customer-care assistant. First, a recall-oriented semantic matcher identifies messages related to a ten-skill domain family without consulting account state. Second, a deterministic executability gate removes candidates whose internal hard-stop conditions hold. Because the gate and the skill evaluate the same exit predicates, every blocked candidate would be unable to complete under the same account state, provided predicate parity is preserved and both checks observe fresh authoritative state. Finally, the LLM decides whether to invoke one of the remaining candidates. In a post-launch production analysis of 756.6K user messages across 267.6K conversations, semantic matching retained 174,927 messages (23.1%). Within this matched stream, the gate removed 1,039,462 of 1,749,270 skill-message pairs (59.4%), saving 228.8 million skill-description tokens -- 59.1% of the post-semantic skill-description footprint. Together, semantic matching and executability gating reduced skill-description context by 90.5% relative to exposing all ten skills to every message. To test whether this pruning affects model behavior rather than context size alone, we replayed a risk-enriched cohort of 1,000 conversations with all ten skills exposed. The model selected a production-blocked skill in 78 conversations (7.8%). This counterfactual result shows that deterministic gating prevents non-executable candidates from influencing model selection, while not claiming downstream tool execution or customer-outcome effects.