# Predicting Object Interactions with Behavior Primitives: An Application in Stowing Tasks

arXiv: https://arxiv.org/abs/2309.16873 · HTML: https://mlsanity.com/paper/2309.16873/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2023.

## Built on by (most cited first, 13 of 13)

- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policy](https://mlsanity.com/paper/2410.17488/) (2024)
- [RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing](https://mlsanity.com/paper/2407.01418/) (2024)
- [Think Small, Act Big: Primitive Prompt Learning for Lifelong Robot Manipulation](https://mlsanity.com/paper/2504.00420/) (2025)
- [Learning Coordinated Bimanual Manipulation Policies using State Diffusion and Inverse Dynamics Models](https://mlsanity.com/paper/2503.23271/) (2025)
- [3D Flow Diffusion Policy: Visuomotor Policy Learning via Generating Flow in 3D Space](https://mlsanity.com/paper/2509.18676/) (2025)
- [PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learning](https://mlsanity.com/paper/2403.00929/) (2024)
- [SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models](https://mlsanity.com/paper/2512.05955/) (2025)
- [DriftWorld: Fast World Modeling through Drifting](https://mlsanity.com/paper/2607.15065/) (2026)
- [Structured 4D Latent Predictive Model for Robot Planning](https://mlsanity.com/paper/2607.01166/) (2026)
- [CoStream: Composing Simple Behaviors for Generalizable Complex Manipulation](https://mlsanity.com/paper/2606.26423/) (2026)
- [Visual Foresight for Robotic Stow: A Diffusion-Based World Model from Sparse Snapshots](https://mlsanity.com/paper/2602.13347/) (2026)
- [Robot Learning with Visual Predicted Force](https://mlsanity.com/paper/2610.04741/) (2026)

## Abstract (arXiv)

Stowing, the task of placing objects in cluttered shelves or bins, is a common task in warehouse and manufacturing operations. However, this task is still predominantly carried out by human workers as stowing is challenging to automate due to the complex multi-object interactions and long-horizon nature of the task. Previous works typically involve extensive data collection and costly human labeling of semantic priors across diverse object categories. This paper presents a method to learn a generalizable robot stowing policy from predictive model of object interactions and a single demonstration with behavior primitives. We propose a novel framework that utilizes Graph Neural Networks to predict object interactions within the parameter space of behavioral primitives. We further employ primitive-augmented trajectory optimization to search the parameters of a predefined library of heterogeneous behavioral primitives to instantiate the control action. Our framework enables robots to proficiently execute long-horizon stowing tasks with a few keyframes (3-4) from a single demonstration. Despite being solely trained in a simulation, our framework demonstrates remarkable generalization capabilities. It efficiently adapts to a broad spectrum of real-world conditions, including various shelf widths, fluctuating quantities of objects, and objects with diverse attributes such as sizes and shapes.