# Reliable Collaborative Conversational Agent System Based on LLMs and Answer Set Programming

arXiv: https://arxiv.org/abs/2505.06438 · HTML: https://mlsanity.com/paper/2505.06438/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 9 May 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [A Categorical Archive of ChatGPT Failures](https://mlsanity.com/paper/2302.03494/) (2023)
- [Constraint Answer Set Programming without Grounding](https://mlsanity.com/paper/1804.11162/) (2018)
- [Justifications for Goal-Directed Constraint Answer Set Programming](https://mlsanity.com/paper/2009.10238/) (2020)
- [A Physician Advisory System for Chronic Heart Failure Management Based on Knowledge Patterns](https://mlsanity.com/paper/1610.08115/) (2016)
- [Reliable Natural Language Understanding with Large Language Models and Answer Set Programming](https://mlsanity.com/paper/2302.03780/) (2023)
- [Automated Interactive Domain-Specific Conversational Agents that Understand Human Dialogs](https://mlsanity.com/paper/2303.08941/) (2023)
- [CASPR: A Commonsense Reasoning-based Conversational Socialbot](https://mlsanity.com/paper/2110.05387/) (2021)
- [A Reliable Common-Sense Reasoning Socialbot Built Using LLMs and Goal-Directed ASP](https://mlsanity.com/paper/2407.18498/) (2024)

## Abstract (arXiv)

As the Large-Language-Model-driven (LLM-driven) Artificial Intelligence (AI) bots became popular, people realized their strong potential in Task-Oriented Dialogue (TOD). However, bots relying wholly on LLMs are unreliable in their knowledge, and whether they can finally produce a correct outcome for the task is not guaranteed. The collaboration among these agents also remains a challenge, since the necessary information to convey is unclear, and the information transfer is by prompts: unreliable, and malicious knowledge is easy to inject. With the help of knowledge representation and reasoning tools such as Answer Set Programming (ASP), conversational agents can be built safely and reliably, and communication among the agents made more reliable as well. We propose a Manager-Customer-Service Dual-Agent paradigm, where ASP-driven bots share the same knowledge base and complete their assigned tasks independently. The agents communicate with each other through the knowledge base, ensuring consistency. The knowledge and information conveyed are encapsulated and invisible to the users, ensuring the security of information transmission. To illustrate the dual-agent conversational paradigm, we have constructed AutoManager, a collaboration system for managing the drive-through window of a fast-food restaurant such as Taco Bell in the US. In AutoManager, the customer service bot takes the customer's order while the manager bot manages the menu and food supply. We evaluated our AutoManager system and compared it with the real-world Taco Bell Drive-Thru AI Order Taker, and the results show that our method is more reliable.