# VERAGMIL: Virtual Environment for Scooping Granular Foods with Imitation Learning Models

arXiv: https://arxiv.org/abs/2608.18258 · HTML: https://mlsanity.com/paper/2608.18258/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 18 Aug 2026.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 21 of 21)

- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder](https://mlsanity.com/paper/1711.00614/) (2017)
- [Text2Reward: Reward Shaping with Language Models for Reinforcement Learning](https://mlsanity.com/paper/2309.11489/) (2023)
- [Assistive Gym: A Physics Simulation Framework for Assistive Robotics](https://mlsanity.com/paper/1910.04700/) (2019)
- [Perceptive Locomotion through Nonlinear Model Predictive Control](https://mlsanity.com/paper/2208.08373/) (2022)
- [Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery](https://mlsanity.com/paper/2406.06092/) (2024)
- [Learning Bimanual Scooping Policies for Food Acquisition](https://mlsanity.com/paper/2211.14652/) (2022)
- [Learning Visuo-Haptic Skewering Strategies for Robot-Assisted Feeding](https://mlsanity.com/paper/2211.14648/) (2022)
- [Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously Unseen Food Items](https://mlsanity.com/paper/1908.07088/) (2019)
- [Leveraging Demonstrations to Improve Online Learning: Quality Matters](https://mlsanity.com/paper/2302.03319/) (2023)
- [VILD: Variational Imitation Learning with Diverse-quality Demonstrations](https://mlsanity.com/paper/1909.06769/) (2019)
- [Representing Robot Geometry as Distance Fields: Applications to Whole-body Manipulation](https://mlsanity.com/paper/2307.00533/) (2023)
- [Leveraging Post Hoc Context for Faster Learning in Bandit Settings with Applications in Robot-Assisted Feeding](https://mlsanity.com/paper/2011.02604/) (2020)
- [Uncertainty-Aware Self-Supervised Target-Mass Grasping of Granular Foods](https://mlsanity.com/paper/2105.12946/) (2021)
- [Assistive VR Gym: Interactions with Real People to Improve Virtual Assistive Robots](https://mlsanity.com/paper/2007.04959/) (2020)
- [Human-Robot Commensality: Bite Timing Prediction for Robot-Assisted Feeding in Groups](https://mlsanity.com/paper/2207.03348/) (2022)
- [Active Robot-Assisted Feeding with a General-Purpose Mobile Manipulator: Design, Evaluation, and Lessons Learned](https://mlsanity.com/paper/1904.03568/) (2019)
- [FF-SRL: High Performance GPU-Based Surgical Simulation For Robot Learning](https://mlsanity.com/paper/2503.18616/) (2025)
- [Frailty Care Robot for Elderly and Its Application for Physical and Psychological Support](https://mlsanity.com/paper/2111.10646/) (2021)

## Abstract (arXiv)

Robot-Assisted Feeding (RAF) systems are essential for assisting individuals with disabilities or motor impairments in eating tasks. Manipulating granular food items, such as rice and beans, poses significant challenges due to their dynamic physical properties. Learning from human demonstrations offers a promising solution, but acquiring high-quality demonstrations is complex. To address this, we present VERAGMIL, a framework that combines a high-fidelity simulator with an intuitive Virtual Reality (VR) interface for recording demonstrations and supporting different imitation learning methods. VERAGMIL provides a realistic environment for training RAF systems to handle granular materials, including robots, sensors, and various food items with distinct physical characteristics. We evaluate VERAGMIL by training three imitation learning models, BC, BC-RNN, and BCQ, on granular scooping and transporting tasks using both VR interface and 3D space mouse demonstrations, comparing them with a human-expert baseline. The models are assessed on success rate, spillage, generalization to unseen food items, and task completion time. Results show that VR-based demonstrations significantly outperform 3D space mouse data, with BCQ achieving the best overall performance, particularly in reducing spillage and approaching human performance. These findings underscore the effectiveness of our framework for training RAF systems in granular material handling. The code for our framework is publicly available at: https://github.com/AmanuelErgogo/VERAGMIL.git.