World-Model-Based Control for Industrial box-packing of Multiple Objects using NewtonianVAE
On the map: World models → Embodied and robot world models
Abstract (arXiv)
The process of industrial box-packing, which involves the accurate placement of multiple objects, requires high-accuracy positioning and sequential actions. When a robot is tasked with placing an object at a specific location with high accuracy, it is important not only to have information about the location of the object to be placed, but also the posture of the object grasped by the robotic hand. Often, industrial box-packing requires the sequential placement of identically shaped objects into a single box. The robot's action should be determined by the same learned model. In factories, new kinds of products often appear and there is a need for a model that can easily adapt to them. Therefore, it should be easy to collect data to train the model. In this study, we designed a robotic system to automate real-world industrial tasks, employing a vision-based learning control model. We propose in-hand-view-sensitive Newtonian variational autoencoder (ihVS-NVAE), which employs an RGB camera to obtain in-hand postures of objects. We demonstrate that our model, trained for a single object-placement task, can handle sequential tasks without additional training. To evaluate efficacy of the proposed model, we employed a real robot to perform sequential industrial box-packing of multiple objects. Results showed that the proposed model achieved a 100% success rate in industrial box-packing tasks, thereby outperforming the state-of-the-art and conventional approaches, underscoring its superior effectiveness and potential in industrial tasks.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 16 of its 21 references that are in the map. First posted on arXiv 4 Aug 2023.
Builds on
16 of 21 references.
References · 16
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
- 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
- 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
- 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
- 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
- 2015Learning Contact-Rich Manipulation Skills with Guided Policy Searchcited by 37
- 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
- 2020PlaNet of the Bayesians: Reconsidering and Improving Deep Planning Network by Incorporating Bayesian Inferencecited by 22
4 more
- 2020NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spacescited by 15
- 2019Tactile-Based Insertion for Dense Box-Packingcited by 12
- 2020Domain-Adversarial and Conditional State Space Model for Imitation Learningcited by 7
- 2022Multi-View Dreaming: Multi-View World Model with Contrastive Learningcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2022Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector Insertion
- 2019Contact Skill Imitation Learning for Robot-Independent Assembly Programming
- 2023Multi-View Masked World Models for Visual Robotic Manipulation
- 2026Generalizable Robotic Insertion with World Models
- 2025IGen: Scalable Data Generation for Robot Learning from Open-World Images
- 2022RT-1: Robotics Transformer for Real-World Control at Scale
- 2026GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks
- 2025Manual2Skill: Learning to Read Manuals and Acquire Robotic Skills for Furniture Assembly Using Vision-Language Models