# Towards an LLM-Based Speech Interface for Robot-Assisted Feeding

arXiv: https://arxiv.org/abs/2410.20624 · HTML: https://mlsanity.com/paper/2410.20624/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 27 Oct 2024.

## Builds on (most cited first, 10 of 10)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [Real-World Robot Applications of Foundation Models: A Review](https://mlsanity.com/paper/2402.05741/) (2024)
- [Shared Autonomy via Hindsight Optimization for Teleoperation and Teaming](https://mlsanity.com/paper/1706.00155/) (2017)
- [Generative Expressive Robot Behaviors using Large Language Models](https://mlsanity.com/paper/2401.14673/) (2024)
- [VoicePilot: Harnessing LLMs as Speech Interfaces for Physically Assistive Robots](https://mlsanity.com/paper/2404.04066/) (2024)
- [Independence in the Home: A Wearable Interface for a Person with Quadriplegia to Teleoperate a Mobile Manipulator](https://mlsanity.com/paper/2312.15071/) (2023)
- [GRID: A Platform for General Robot Intelligence Development](https://mlsanity.com/paper/2310.00887/) (2023)
- [High-density Electromyography for Effective Gesture-based Control of Physically Assistive Mobile Manipulators](https://mlsanity.com/paper/2312.07745/) (2023)

## Built on by (most cited first, 2 of 2)

- [Benchmarking LLM Privacy Recognition for Social Robot Decision Making](https://mlsanity.com/paper/2507.16124/) (2025)
- [UWBench: A Comprehensive Vision-Language Benchmark for Underwater Understanding](https://mlsanity.com/paper/2510.18262/) (2025)

## Abstract (arXiv)

Physically assistive robots present an opportunity to significantly increase the well-being and independence of individuals with motor impairments or other forms of disability who are unable to complete activities of daily living (ADLs). Speech interfaces, especially ones that utilize Large Language Models (LLMs), can enable individuals to effectively and naturally communicate high-level commands and nuanced preferences to robots. In this work, we demonstrate an LLM-based speech interface for a commercially available assistive feeding robot. Our system is based on an iteratively designed framework, from the paper "VoicePilot: Harnessing LLMs as Speech Interfaces for Physically Assistive Robots," that incorporates human-centric elements for integrating LLMs as interfaces for robots. It has been evaluated through a user study with 11 older adults at an independent living facility. Videos are located on our project website: https://sites.google.com/andrew.cmu.edu/voicepilot/.