# GPT Models Meet Robotic Applications: Co-Speech Gesturing Chat System

arXiv: https://arxiv.org/abs/2306.01741 · HTML: https://mlsanity.com/paper/2306.01741/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 9 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 4 of 4)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [ChatGPT Empowered Long-Step Robot Control in Various Environments: A Case Application](https://mlsanity.com/paper/2304.03893/) (2023)
- [Design of conversational humanoid robot based on hardware independent gesture generation](https://mlsanity.com/paper/1905.08702/) (2019)
- [Labeling the Phrases of a Conversational Agent with a Unique Personalized Vocabulary](https://mlsanity.com/paper/2010.06194/) (2020)

## Built on by (most cited first, 4 of 4)

- [Agent AI: Surveying the Horizons of Multimodal Interaction](https://mlsanity.com/paper/2401.03568/) (2024)
- [An Interactive Agent Foundation Model](https://mlsanity.com/paper/2402.05929/) (2024)
- [How Can Large Language Models Enable Better Socially Assistive Human-Robot Interaction: A Brief Survey](https://mlsanity.com/paper/2404.00938/) (2024)
- [Position Paper: Agent AI Towards a Holistic Intelligence](https://mlsanity.com/paper/2403.00833/) (2024)

## Abstract (arXiv)

This technical paper introduces a chatting robot system that utilizes recent advancements in large-scale language models (LLMs) such as GPT-3 and ChatGPT. The system is integrated with a co-speech gesture generation system, which selects appropriate gestures based on the conceptual meaning of speech. Our motivation is to explore ways of utilizing the recent progress in LLMs for practical robotic applications, which benefits the development of both chatbots and LLMs. Specifically, it enables the development of highly responsive chatbot systems by leveraging LLMs and adds visual effects to the user interface of LLMs as an additional value. The source code for the system is available on GitHub for our in-house robot (https://github.com/microsoft/LabanotationSuite/tree/master/MSRAbotChatSimulation) and GitHub for Toyota HSR (https://github.com/microsoft/GPT-Enabled-HSR-CoSpeechGestures).