# CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs

arXiv: https://arxiv.org/abs/2404.01343 · HTML: https://mlsanity.com/paper/2404.01343/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 31 Mar 2024.

## Builds on (most cited first, 10 of 10)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory](https://mlsanity.com/paper/2306.03901/) (2023)

## Built on by (most cited first, 17 of 17)

- [Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models](https://mlsanity.com/paper/2503.02623/) (2025)
- [The Role of Deductive and Inductive Reasoning in Large Language Models](https://mlsanity.com/paper/2410.02892/) (2024)
- [ChatMotion: A Multimodal Multi-Agent for Human Motion Analysis](https://mlsanity.com/paper/2502.18180/) (2025)
- [Bayesian Optimization for Controlled Image Editing via LLMs](https://mlsanity.com/paper/2502.18116/) (2025)
- [CoPL: Collaborative Preference Learning for Personalizing LLMs](https://mlsanity.com/paper/2503.01658/) (2025)
- [Effective Red-Teaming of Policy-Adherent Agents](https://mlsanity.com/paper/2506.09600/) (2025)
- [PosterMate: Audience-driven Collaborative Persona Agents for Poster Design](https://mlsanity.com/paper/2507.18572/) (2025)
- [How Social is It? A Benchmark for LLMs' Capabilities in Multi-user Multi-turn Social Agent Tasks](https://mlsanity.com/paper/2505.04628/) (2025)
- [TongSIM: A General Platform for Simulating Intelligent Machines](https://mlsanity.com/paper/2512.20206/) (2025)
- [T2 of Thoughts: Temperature Tree Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2405.14075/) (2024)
- [CNSocialDepress: A Chinese Social Media Dataset for Depression Risk Detection and Structured Analysis](https://mlsanity.com/paper/2510.11233/) (2025)
- [Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Models](https://mlsanity.com/paper/2601.03265/) (2025)
- [Help Without Being Asked: A Deployed Proactive Agent System for On-Call Support with Continuous Self-Improvement](https://mlsanity.com/paper/2604.09579/) (2026)
- [From Words to Workflows: Automating Business Processes](https://mlsanity.com/paper/2412.03446/) (2024)
- [A Conceptual Framework for AI Capability Evaluations](https://mlsanity.com/paper/2506.18213/) (2025)
- [MindFlow+: A Self-Evolving Agent for E-Commerce Customer Service](https://mlsanity.com/paper/2507.18884/) (2025)
- [Can Small Language Models Handle Context-Summarized Multi-Turn Customer-Service QA? A Synthetic Data-Driven Comparative Evaluation](https://mlsanity.com/paper/2602.00665/) (2026)

## Abstract (arXiv)

Businesses and software platforms are increasingly turning to Large Language Models (LLMs) such as GPT-3.5, GPT-4, GLM-3, and LLaMa-2 for chat assistance with file access or as reasoning agents for customer service. However, current LLM-based customer service models have limited integration with customer profiles and lack the operational capabilities necessary for effective service. Moreover, existing API integrations emphasize diversity over the precision and error avoidance essential in real-world customer service scenarios. To address these issues, we propose an LLM agent named CHOPS (CHat with custOmer Profile in existing System), designed to: (1) efficiently utilize existing databases or systems for accessing user information or interacting with these systems following existing guidelines; (2) provide accurate and reasonable responses or carry out required operations in the system while avoiding harmful operations; and (3) leverage a combination of small and large LLMs to achieve satisfying performance at a reasonable inference cost. We introduce a practical dataset, the CPHOS-dataset, which includes a database, guiding files, and QA pairs collected from CPHOS, an online platform that facilitates the organization of simulated Physics Olympiads for high school teachers and students. We have conducted extensive experiments to validate the performance of our proposed CHOPS architecture using the CPHOS-dataset, with the aim of demonstrating how LLMs can enhance or serve as alternatives to human customer service. Code for our proposed architecture and dataset can be found at {https://github.com/JingzheShi/CHOPS}.