# CCTU: A Benchmark for Tool Use under Complex Constraints

arXiv: https://arxiv.org/abs/2603.15309 · HTML: https://mlsanity.com/paper/2603.15309/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 21 of 21)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [ToolQA: A Dataset for LLM Question Answering with External Tools](https://mlsanity.com/paper/2306.13304/) (2023)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step](https://mlsanity.com/paper/2312.14033/) (2023)
- [COLLIE: Systematic Construction of Constrained Text Generation Tasks](https://mlsanity.com/paper/2307.08689/) (2023)
- [InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences](https://mlsanity.com/paper/2503.11043/) (2025)
- [ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use](https://mlsanity.com/paper/2501.02506/) (2025)
- [AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios](https://mlsanity.com/paper/2505.16944/) (2025)
- [Feedback-Driven Tool-Use Improvements in Large Language Models via Automated Build Environments](https://mlsanity.com/paper/2508.08791/) (2025)
- [Budget-Constrained Tool Learning with Planning](https://mlsanity.com/paper/2402.15960/) (2024)
- [FamilyTool: A Multi-hop Personalized Tool Use Benchmark](https://mlsanity.com/paper/2504.06766/) (2025)

## Built on by (most cited first, 7 of 7)

- [Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents](https://mlsanity.com/paper/2607.10487/) (2026)
- [Many-Tier Instruction Hierarchy in LLM Agents](https://mlsanity.com/paper/2604.09443/) (2026)
- [PeakBench: Benchmarking Resource-Aware Tool Invocation in LLM Agents](https://mlsanity.com/paper/2608.24509/) (2026)
- [RECAP: Regression Evaluation for Continual Adaptation of Prompts](https://mlsanity.com/paper/2606.06698/) (2026)
- [Policy-Masked Private Experts: Auditable and Reversible Capability Access Control in Sparse MoE Models](https://mlsanity.com/paper/2608.06690/) (2026)
- [Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction](https://mlsanity.com/paper/2608.12426/) (2026)
- [MTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding](https://mlsanity.com/paper/2609.14992/) (2026)

## Abstract (arXiv)

Solving problems through tool use under explicit constraints constitutes a highly challenging yet unavoidable scenario for large language models (LLMs), requiring capabilities such as function calling, instruction following, and self-refinement. However, progress has been hindered by the absence of dedicated evaluations. To address this, we introduce CCTU, a benchmark for evaluating LLM tool use under complex constraints. CCTU is grounded in a taxonomy of 12 constraint categories spanning four dimensions (i.e., resource, behavior, toolset, and response). The benchmark comprises 200 carefully curated and challenging test cases across diverse tool-use scenarios, each involving an average of seven constraint types and an average prompt length exceeding 4,700 tokens. To enable reliable evaluation, we develop an executable constraint validation module that performs step-level validation and enforces compliance during multi-turn interactions between models and their environments. We evaluate nine state-of-the-art LLMs in both thinking and non-thinking modes. Results indicate that when strict adherence to all constraints is required, no model achieves a task completion rate above 20%. Further analysis reveals that models violate constraints in over 50% of cases, particularly in the resource and response dimensions. Moreover, LLMs demonstrate limited capacity for self-refinement even after receiving detailed feedback on constraint violations, highlighting a critical bottleneck in the development of robust tool-use agents. To facilitate future research, we release the data and code.