# LAVA: Long-horizon Visual Action based Food Acquisition

arXiv: https://arxiv.org/abs/2403.12876 · HTML: https://mlsanity.com/paper/2403.12876/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2024.

## Built on by (most cited first, 10 of 10)

- [FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalization](https://mlsanity.com/paper/2506.14968/) (2025)
- [PLANRL: A Motion Planning and Imitation Learning Framework to Bootstrap Reinforcement Learning](https://mlsanity.com/paper/2408.04054/) (2024)
- [CAML: Collaborative Auxiliary Modality Learning for Multi-Agent Systems](https://mlsanity.com/paper/2502.17821/) (2025)
- [IMRL: Integrating Visual, Physical, Temporal, and Geometric Representations for Enhanced Food Acquisition](https://mlsanity.com/paper/2409.12092/) (2024)
- [REPeat: A Real2Sim2Real Approach for Pre-acquisition of Soft Food Items in Robot-assisted Feeding](https://mlsanity.com/paper/2410.10017/) (2024)
- [Adaptive Conformal Guidance for Learning under Uncertainty](https://mlsanity.com/paper/2502.16736/) (2025)
- [MMCD: Multi-Modal Collaborative Decision-Making for Connected Autonomy with Knowledge Distillation](https://mlsanity.com/paper/2509.18198/) (2025)
- [SCOOP'D: Learning Mixed-Liquid-Solid Scooping via Sim2Real Generative Policy](https://mlsanity.com/paper/2510.11566/) (2025)
- [PRISM: Performer RS-IMLE for Single-pass Multisensory Imitation Learning](https://mlsanity.com/paper/2602.02396/) (2026)
- [GRITS: A Spillage-Aware Guided Diffusion Policy for Robot Food Scooping Tasks](https://mlsanity.com/paper/2510.00573/) (2025)

## Abstract (arXiv)

Robotic Assisted Feeding (RAF) addresses the fundamental need for individuals with mobility impairments to regain autonomy in feeding themselves. The goal of RAF is to use a robot arm to acquire and transfer food to individuals from the table. Existing RAF methods primarily focus on solid foods, leaving a gap in manipulation strategies for semi-solid and deformable foods. This study introduces Long-horizon Visual Action (LAVA) based food acquisition of liquid, semisolid, and deformable foods. Long-horizon refers to the goal of "clearing the bowl" by sequentially acquiring the food from the bowl. LAVA employs a hierarchical policy for long-horizon food acquisition tasks. The framework uses high-level policy to determine primitives by leveraging ScoopNet. At the mid-level, LAVA finds parameters for primitives using vision. To carry out sequential plans in the real world, LAVA delegates action execution which is driven by Low-level policy that uses parameters received from mid-level policy and behavior cloning ensuring precise trajectory execution. We validate our approach on complex real-world acquisition trials involving granular, liquid, semisolid, and deformable food types along with fruit chunks and soup acquisition. Across 46 bowls, LAVA acquires much more efficiently than baselines with a success rate of 89 +/- 4% and generalizes across realistic plate variations such as different positions, varieties, and amount of food in the bowl. Code, datasets, videos, and supplementary materials can be found on our website.