# Human-Robot Commensality: Bite Timing Prediction for Robot-Assisted Feeding in Groups

arXiv: https://arxiv.org/abs/2207.03348 · HTML: https://mlsanity.com/paper/2207.03348/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 75 references that are in the map.
- First posted on arXiv 7 Jul 2022.

## Builds on (most cited first, 11 of 11)

- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [Temporal Convolutional Networks: A Unified Approach to Action Segmentation](https://mlsanity.com/paper/1608.08242/) (2016)
- [Multi-Person 3D Motion Prediction with Multi-Range Transformers](https://mlsanity.com/paper/2111.12073/) (2021)
- [Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interaction](https://mlsanity.com/paper/1906.04158/) (2019)
- [Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulation](https://mlsanity.com/paper/1804.08768/) (2018)
- [Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding](https://mlsanity.com/paper/2011.02604/) (2020)
- [Balancing Efficiency and Comfort in Robot-Assisted Bite Transfer](https://mlsanity.com/paper/2111.11401/) (2021)
- [Towards Assistive Feeding with a General-Purpose Mobile Manipulator](https://mlsanity.com/paper/1605.07996/) (2016)
- [Effects of Interruptibility-Aware Robot Behavior](https://mlsanity.com/paper/1804.06383/) (2018)

## Built on by (most cited first, 5 of 5)

- [GRACE: Generalizing Robot-Assisted Caregiving with User Functionality Embeddings](https://mlsanity.com/paper/2501.17855/) (2025)
- [REPeat: A Real2Sim2Real Approach for Pre-acquisition of Soft Food Items in Robot-assisted Feeding](https://mlsanity.com/paper/2410.10017/) (2024)
- [Robot-Assisted Social Dining as a White Glove Service](https://mlsanity.com/paper/2602.15767/) (2026)
- [VERAGMIL: Virtual Environment for Scooping Granular Foods with Imitation Learning Models](https://mlsanity.com/paper/2608.18258/) (2026)
- [STARS: From Spatiotemporal Dynamics to Social Representations in Human-Robot Interaction](https://mlsanity.com/paper/2609.40245/) (2026)

## Abstract (arXiv)

We develop data-driven models to predict when a robot should feed during social dining scenarios. Being able to eat independently with friends and family is considered one of the most memorable and important activities for people with mobility limitations. While existing robotic systems for feeding people with mobility limitations focus on solitary dining, commensality, the act of eating together, is often the practice of choice. Sharing meals with others introduces the problem of socially appropriate bite timing for a robot, i.e. the appropriate timing for the robot to feed without disrupting the social dynamics of a shared meal. Our key insight is that bite timing strategies that take into account the delicate balance of social cues can lead to seamless interactions during robot-assisted feeding in a social dining scenario. We approach this problem by collecting a Human-Human Commensality Dataset (HHCD) containing 30 groups of three people eating together. We use this dataset to analyze human-human commensality behaviors and develop bite timing prediction models in social dining scenarios. We also transfer these models to human-robot commensality scenarios. Our user studies show that prediction improves when our algorithm uses multimodal social signaling cues between diners to model bite timing. The HHCD dataset, videos of user studies, and code are available at https://emprise.cs.cornell.edu/hrcom/