mlsanityData to 14 Oct 2026

Harnessing LLMs for API Interactions: A Framework for Classification and Synthetic Data Generation

arXiv 2409.11703 ↗First posted 18 Sep 2024cs.CLcited by 17

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

As Large Language Models (LLMs) advance in natural language processing, there is growing interest in leveraging their capabilities to simplify software interactions. In this paper, we propose a novel system that integrates LLMs for both classifying natural language inputs into corresponding API calls and automating the creation of sample datasets tailored to specific API functions. By classifying natural language commands, our system allows users to invoke complex software functionalities through simple inputs, improving interaction efficiency and lowering the barrier to software utilization. Our dataset generation approach also enables the efficient and systematic evaluation of different LLMs in classifying API calls, offering a practical tool for developers or business owners to assess the suitability of LLMs for customized API management. We conduct experiments on several prominent LLMs using generated sample datasets for various API functions. The results show that GPT-4 achieves a high classification accuracy of 0.996, while LLaMA-3-8B performs much worse at 0.759. These findings highlight the potential of LLMs to transform API management and validate the effectiveness of our system in guiding model testing and selection across diverse applications.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 28 of its 45 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

28 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey14 Jun

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023BloombergGPT: A Large Language Model for Financecited by 939
  5. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  6. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  7. 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 135
  8. 2022Can Language Understand Depth?cited by 69
  9. 2023A Text-guided Protein Design Frameworkcited by 64
  10. 2024ComposerX: Multi-Agent Symbolic Music Composition with LLMscited by 42
  11. 2024WorDepth: Variational Language Prior for Monocular Depth Estimationcited by 34
  12. 2019The FacT: Taming Latent Factor Models for Explainability with Factorization Treescited by 27
15 more
  1. 2024RAG-RLRC-LaySum at BioLaySumm: Integrating Retrieval-Augmented Generation and Readability Control for Layman Summarization of Biomedical Textscited by 23
  2. 2023Diffusion Modeling with Domain-conditioned Prior Guidance for Accelerated MRI and qMRI Reconstructioncited by 22
  3. 2024Prototypical Reward Network for Data-Efficient RLHFcited by 22
  4. 2024Towards Resilient and Efficient LLMs: A Comparative Study of Efficiency, Performance, and Adversarial Robustnesscited by 22
  5. 2024Identification of Prognostic Biomarkers for Stage III Non-Small Cell Lung Carcinoma in Female Nonsmokers Using Machine Learningcited by 17
  6. 2024TFWT: Tabular Feature Weighting with Transformercited by 12
  7. 2023Prediction of COVID-19 Patients' Emergency Room Revisit using Multi-Source Transfer Learningcited by 11
  8. 2024Graphical Structural Learning of rs-fMRI data in Heavy Smokerscited by 11
  9. 2021An Optimal Control Framework for Joint-channel Parallel MRI Reconstruction without Coil Sensitivitiescited by 10
  10. 2024Rhyme-aware Chinese lyric generator based on GPTcited by 10
  11. 2024Online Learning of Multiple Tasks and Their Relationships : Testing on Spam Email Data and EEG Signals Recorded in Construction Fieldscited by 8
  12. 2024Hedge Fund Portfolio Construction Using PolyModel Theory and iTransformercited by 7
  13. 2024LLM-based Frameworks for API Argument Filling in Task-Oriented Conversational Systemscited by 4
  14. 2024Exploring the Generalization of Cancer Clinical Trial Eligibility Classifiers Across Diseasescited by 3
  15. 2023Automated DevOps Pipeline Generation for Code Repositories using Large Language Modelscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 13; 2025: 4.

17 papers.

Sort:

By area:

  1. 2025GTM: Simulating the World of Tools for AI Agentscited by 14
  2. 2024Adaptive Optimization for Enhanced Efficiency in Large-Scale Language Model Trainingcited by 13
  3. 2024CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharingcited by 12
  4. 2024LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuningcited by 10
  5. 2024Multi-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICscited by 10
  6. 2024Enhanced Recommendation Combining Collaborative Filtering and Large Language Modelscited by 10
  7. 2024Leveraging Convolutional Neural Network-Transformer Synergy for Predictive Modeling in Risk-Based Applicationscited by 9
  8. 2024AI-Driven Health Monitoring of Distributed Computing Architecture: Insights from XGBoost and SHAPcited by 7
  9. 2025Evaluation of LLMs for mathematical problem solvingcited by 7
  10. 2025DeepSeek-Inspired Exploration of RL-based LLMs and Synergy with Wireless Networks: A Surveycited by 6
  11. 2024Optimizing Multi-Task Learning for Enhanced Performance in Large Language Modelscited by 5
  12. 2024Collaborative Optimization in Financial Data Mining Through Deep Learning and ResNeXtcited by 5
5 more
  1. 2024Feature Alignment-Based Knowledge Distillation for Efficient Compression of Large Language Modelscited by 5
  2. 2024Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Taskscited by 4
  3. 2024Computer Vision-Driven Gesture Recognition: Toward Natural and Intuitive Human-Computercited by 3
  4. 2024Dynamic User Interface Generation for Enhanced Human-Computer Interaction Using Variational Autoencoderscited by 2
  5. 2025LLM Agents for Generating Microservice-based Applications: how complex is your specification?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs
  2. 2025Evaluating LLMs on Sequential API Call Through Automated Test Generation
  3. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
  4. 2024Octopus: On-device language model for function calling of software APIs
  5. 2024Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs
  6. 2026When LLMs Lag Behind: Knowledge Conflicts from Evolving APIs in Code Generation
  7. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets
  8. 2024A Comprehensive Framework for Evaluating API-oriented Code Generation in Large Language Models