# Enhancing Supermarket Robot Interaction: A Multi-Level LLM Conversational Interface for Handling Diverse Customer Intents

arXiv: https://arxiv.org/abs/2406.11047 · HTML: https://mlsanity.com/paper/2406.11047/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 16 Jun 2024.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [RAFT: Adapting Language Model to Domain Specific RAG](https://mlsanity.com/paper/2403.10131/) (2024)
- [Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment](https://mlsanity.com/paper/2312.12148/) (2023)
- [Achieving Human Parity in Conversational Speech Recognition](https://mlsanity.com/paper/1610.05256/) (2016)
- [In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss](https://mlsanity.com/paper/2402.10790/) (2024)
- [Bias in Automated Speaker Recognition](https://mlsanity.com/paper/2201.09486/) (2022)
- [Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses](https://mlsanity.com/paper/2312.00763/) (2023)
- [AequeVox: Automated Fairness Testing of Speech Recognition Systems](https://mlsanity.com/paper/2110.09843/) (2021)
- [Task-Level Authoring for Remote Robot Teleoperation](https://mlsanity.com/paper/2109.02301/) (2021)
- [WhisperWand: Simultaneous Voice and Gesture Tracking Interface](https://mlsanity.com/paper/2301.10314/) (2023)

## Built on by (most cited first, 5 of 5)

- [PersonaGym: Evaluating Persona Agents and LLMs](https://mlsanity.com/paper/2407.18416/) (2024)
- [Examining Identity Drift in Conversations of LLM Agents](https://mlsanity.com/paper/2412.00804/) (2024)
- [Robots Can Multitask Too: Integrating a Memory Architecture and LLMs for Enhanced Cross-Task Robot Action Generation](https://mlsanity.com/paper/2407.13505/) (2024)
- [GE-Chat: A Graph Enhanced RAG Framework for Evidential Response Generation of LLMs](https://mlsanity.com/paper/2505.10143/) (2025)
- [MindFlow+: A Self-Evolving Agent for E-Commerce Customer Service](https://mlsanity.com/paper/2507.18884/) (2025)

## Abstract (arXiv)

This paper presents the design and evaluation of a novel multi-level LLM interface for supermarket robots to assist customers. The proposed interface allows customers to convey their needs through both generic and specific queries. While state-of-the-art systems like OpenAI's GPTs are highly adaptable and easy to build and deploy, they still face challenges such as increased response times and limitations in strategic control of the underlying model for tailored use-case and cost optimization. Driven by the goal of developing faster and more efficient conversational agents, this paper advocates for using multiple smaller, specialized LLMs fine-tuned to handle different user queries based on their specificity and user intent. We compare this approach to a specialized GPT model powered by GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) and qualitative participant feedback in a counterbalanced within-subjects experiment. Our findings show that our multi-LLM chatbot architecture outperformed the benchmarked GPT model across all 13 measured criteria, with statistically significant improvements in four key areas: performance, user satisfaction, user-agent partnership, and self-image enhancement. The paper also presents a method for supermarket robot navigation by mapping the final chatbot response to correct shelf numbers, enabling the robot to sequentially navigate towards the respective products, after which lower-level robot perception, control, and planning can be used for automated object retrieval. We hope this work encourages more efforts into using multiple, specialized smaller models instead of relying on a single powerful, but more expensive and slower model.