mlsanityData to 7 Oct 2026

Advancing and Benchmarking Personalized Tool Invocation for LLMs

arXiv 2505.04072 ↗First posted 7 May 2025cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Tool invocation is a crucial mechanism for extending the capabilities of Large Language Models (LLMs) and has recently garnered significant attention. It enables LLMs to solve complex problems through tool calls while accessing up-to-date world knowledge. However, existing work primarily focuses on the fundamental ability of LLMs to invoke tools for problem-solving, without considering personalized constraints in tool invocation. In this work, we introduce the concept of Personalized Tool Invocation and define two key tasks: Tool Preference and Profile-dependent Query. Tool Preference addresses user preferences when selecting among functionally similar tools, while Profile-dependent Query considers cases where a user query lacks certain tool parameters, requiring the model to infer them from the user profile. To tackle these challenges, we propose PTool, a data synthesis framework designed for personalized tool invocation. Additionally, we construct PTBench, the first benchmark for evaluating personalized tool invocation. We then fine-tune various open-source models, demonstrating the effectiveness of our framework and providing valuable insights. Our benchmark is public at https://github.com/hyfshadow/PTBench.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 7 May 2025.

Builds on

33 of 46 references.

References · 33

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  9. 2020Aligning AI With Shared Human Valuescited by 706
  10. 2023A Survey on Large Language Models for Recommendationcited by 378
  11. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
  12. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
21 more
  1. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 245
  2. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  3. 2024Tool Learning with Large Language Models: A Surveycited by 221
  4. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
  5. 2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 171
  6. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  7. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
  8. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systemscited by 108
  9. 2024Personalization of Large Language Models: A Surveycited by 100
  10. 2023EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Educationcited by 80
  11. 2023ControlLLM: Augment Language Models with Tools by Searching on Graphscited by 59
  12. 2024Learning to Use Tools via Cooperative and Interactive Agentscited by 54
  13. 2024AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learningcited by 53
  14. 2023Multimodal LLMs for health grounded in individual-specific datacited by 48
  15. 2023A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3cited by 46
  16. 2023Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insightscited by 38
  17. 2024Empowering Personalized Learning through a Conversation-based Tutoring System with Student Modelingcited by 33
  18. 2023LLM4Vis: Explainable Visualization Recommendation using ChatGPTcited by 25
  19. 2023RecExplainer: Aligning Large Language Models for Explaining Recommendation Modelscited by 14
  20. 2023Can LLMs be Good Financial Advisors?: An Initial Study in Personal Decision Making for Optimized Outcomescited by 10
  21. 2024Knowledge-Infused LLM-Powered Conversational Health Agent: A Case Study for Diabetes Patientscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025GTM: Simulating the World of Tools for AI Agentscited by 12
  2. 2026Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directionscited by 10
  3. 2026Memory-Induced Tool-Drift in LLM Agentscited by 4
  4. 2026Latent Preference Modeling for Multi-Session Personalized Tool Callingcited by 2
  5. 2025ColorEcosystem: Powering Personalized, Standardized, and Trustworthy Agentic Service in massive-agent Ecosystemcited by 0
  6. 2026UserToolBench: A User-Profile-Hidden Benchmark for Personalized Decision Making in Tool-Use LLMscited by 0
  7. 2026PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agentscited by 0