REPeat: A Real2Sim2Real Approach for Pre-acquisition of Soft Food Items in Robot-assisted Feeding
Not on the map. Why?
Abstract (arXiv)
The paper presents REPeat, a Real2Sim2Real framework designed to enhance bite acquisition in robot-assisted feeding for soft foods. It uses `pre-acquisition actions' such as pushing, cutting, and flipping to improve the success rate of bite acquisition actions such as skewering, scooping, and twirling. If the data-driven model predicts low success for direct bite acquisition, the system initiates a Real2Sim phase, reconstructing the food's geometry in a simulation. The robot explores various pre-acquisition actions in the simulation, then a Sim2Real step renders a photorealistic image to reassess success rates. If the success improves, the robot applies the action in reality. We evaluate the system on 15 diverse plates with 10 types of food items for a soft food diet, showing improvement in bite acquisition success rates by 27% on average across all plates. See our project website at https://emprise.cs.cornell.edu/repeat.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 57 references that are in the map. First posted on arXiv 13 Oct 2024.
Builds on
29 of 57 references.
References · 29
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2015Fast R-CNNcited by 2,930
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2021PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physicscited by 100
- 2020Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learningcited by 69
- 2021DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cuttingcited by 62
- 2020Visual Grounding of Learned Physical Modelscited by 54
- 2022DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Toolscited by 44
- 2022Predictive Sampling: Real-time Behaviour Synthesis with MuJoCocited by 39
- 2023FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulationcited by 37
- 2020LagNetViP: A Lagrangian Neural Network for Video Predictioncited by 24
17 more
- 2022Learning Bimanual Scooping Policies for Food Acquisitioncited by 19
- 2018Towards Robotic Feeding: Role of Haptics in Fork-based Food Manipulationcited by 16
- 2019Leveraging Multimodal Haptic Sensory Data for Robust Cuttingcited by 15
- 2022Learning Visuo-Haptic Skewering Strategies for Robot-Assisted Feedingcited by 15
- 2019Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously Unseen Food Itemscited by 14
- 2022RCareWorld: A Human-centric Simulation World for Caregiving Robotscited by 14
- 2022It Takes Two: Learning to Plan for Human-Robot Cooperative Carryingcited by 12
- 2020Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feedingcited by 11
- 2024LAVA: Long-horizon Visual Action based Food Acquisitioncited by 10
- 2024MORPHeus: a Multimodal One-armed Robot-assisted Peeling System with Human Users In-the-loopcited by 10
- 2024FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishescited by 10
- 2024Feel the Bite: Robot-Assisted Inside-Mouth Bite Transfer using Robust Mouth Perception and Physical Interaction-Aware Controlcited by 9
- 2024Adaptive Visual Imitation Learning for Robotic Assisted Feeding Across Varied Bowl Configurations and Food Typescited by 8
- 2023Learning Sequential Acquisition Policies for Robot-Assisted Feedingcited by 6
- 2022Human-Robot Commensality: Bite Timing Prediction for Robot-Assisted Feeding in Groupscited by 5
- 2022SPARCS: Structuring Physically Assistive Robotics for Caregiving with Stakeholders-in-the-loopcited by 5
- 2019Robot-Assisted Feeding: Generalizing Skewering Strategies across Food Items on a Realistic Platecited by 4
Built on by
Citing papers per year: 2025: 2.
2 papers.