mlsanityData to 7 Oct 2026

Automated test generation to evaluate tool-augmented LLMs as conversational AI agents

arXiv 2409.15934 ↗First posted 24 Sep 2024cs.CL cs.AI cs.LGcited by 17

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Tool-augmented LLMs are a promising approach to create AI agents that can have realistic conversations, follow procedures, and call appropriate functions. However, evaluating them is challenging due to the diversity of possible conversations, and existing datasets focus only on single interactions and function-calling. We present a test generation pipeline to evaluate LLMs as conversational AI agents. Our framework uses LLMs to generate diverse tests grounded on user-defined procedures. For that, we use intermediate graphs to limit the LLM test generator's tendency to hallucinate content that is not grounded on input procedures, and enforces high coverage of the possible conversations. Additionally, we put forward ALMITA, a manually curated dataset for evaluating AI agents in customer support, and use it to evaluate existing LLMs. Our results show that while tool-augmented LLMs perform well in single interactions, they often struggle to handle complete conversations. While our focus is on customer support, our method is general and capable of AI agents for different domains.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 19 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

14 of 19 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security10 Jan
  2. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct

Other references · 12

  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  4. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  5. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  6. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  7. 2023Language is All a Graph Needscited by 219
  8. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
  9. 2024Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Codecited by 106
  10. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
  11. 2024A Complete Survey on LLM-based AI Chatbotscited by 71
  12. 2022Conditional Generation with a Question-Answering Blueprintcited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 6.

17 papers.

Sort:

By area:

  1. 2025Survey on Evaluation of LLM-based Agentscited by 190
  2. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 158
  3. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  4. 2025IntellAgent: A Multi-Agent Framework for Evaluating Conversational AI Systemscited by 25
  5. 2025Pre-Act: Multi-Step Planning and Reasoning Improves Acting in LLM Agentscited by 20
  6. 2025LoopTool: Closing the Data-Training Loop for Robust LLM Tool Callscited by 16
  7. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 12
  8. 2025Delegated Authorization for Agents Constrained to Semantic Task-to-Scope Matchingcited by 5
  9. 2026EMPA: Evaluating Persona-Aligned Empathy as a Processcited by 4
  10. 2025Agent-Testing Agent: A Meta-Agent for Automated Testing and Evaluation of Conversational AI Agentscited by 2
  11. 2026User-Oriented Multi-Turn Dialogue Generation with Tool Use at scalecited by 2
  12. 2025When Users Are Happy but Agents Are Wrong: Multi-Dimensional Evaluation of Tool-Augmented Dialoguecited by 1
5 more
  1. 2026Many Hands Make Light Work: An LLM-based Multi-Agent System for Detecting Malicious PyPI Packagescited by 1
  2. 2025When Actions Teach You to Think: Reasoning-Action Synergy via Reinforcement Learning in Conversational Agentscited by 0
  3. 2026MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Mapscited by 0
  4. 2026Clueing up LLMs with Tool-Augmented Deductive Reasoningcited by 0
  5. 2026Designing Reliable LLM-as-a-Judge Measurement Systems for Multi-Turn Business Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ChainBuddy: An AI Agent System for Generating LLM Pipelines
  2. 2023Towards Autonomous Testing Agents via Conversational Large Language Models
  3. 2026Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows
  4. 2026State-Grounded Multi-Agent Synthetic Data Generation for Tool-Augmented LLMs
  5. 2026Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications
  6. 2026Operationalising Multi-Dimensional Evaluation for Conversational Agents: A Scalable, Governed Pipeline with Selective Re-evaluation and Model Benchmarking
  7. 2025Configurable multi-agent framework for scalable and realistic testing of llm-based agents
  8. 2024Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs