Design and testing of an agent chatbot supporting decision making with public transport data
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Assessing the quality of public transportation services requires the analysis of large quantities of data on the scheduled and actual trips and documents listing the quality constraints each service needs to meet. Interrogating such datasets with SQL queries, organizing and visualizing the data can be quite complex for most users. This paper presents a chatbot offering a user-friendly tool to interact with these datasets and support decision making. It is based on an agent architecture, which expands the capabilities of the core Large Language Model (LLM) by allowing it to interact with a series of tools that can execute several tasks, like performing SQL queries, plotting data and creating maps from the coordinates of a trip and its stops. This paper also tackles one of the main open problems of such Generative AI projects: collecting data to measure the system's performance. Our chatbot has been extensively tested with a workflow that asks several questions and stores the generated query, the retrieved data and the natural language response for each of them. Such questions are drawn from a set of base examples which are then completed with actual data from the database. This procedure yields a dataset for the evaluation of the chatbot's performance, especially the consistency of its answers and the correctness of the generated queries.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 2 of its 14 references that are in the map. First posted on arXiv 28 May 2025.
Builds on
2 of 14 references.
References · 2
- 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQLcited by 162
- 2024Large Language Model Enhanced Text-to-SQL Generation: A Surveycited by 24
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024CHATATC: Large Language Model-Driven Conversational Agents for Supporting Strategic Air Traffic Flow Management
- 2024Traffic Performance GPT (TP-GPT): Real-Time Data Informed Intelligent ChatBot for Transportation Surveillance and Management
- 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey
- 2025MegaChat: A Synthetic Persian Q&A Dataset for High-Quality Sales Chatbot Evaluation
- 2024Automated test generation to evaluate tool-augmented LLMs as conversational AI agents
- 2025Modeling realistic human behavior using generative agents in a multimodal transport system: Software architecture and Application to Toulouse
- 2024AGENTiGraph: An Interactive Knowledge Graph Platform for LLM-based Chatbots Utilizing Private Data
- 2024MAG-V: A Multi-Agent Framework for Synthetic Data Generation and Verification