GeMuCo: Generalized Multisensory Correlational Model for Body Schema Learning
Not on the map. Why?
Abstract (arXiv)
Humans can autonomously learn the relationship between sensation and motion in their own bodies, estimate and control their own body states, and move while continuously adapting to the current environment. On the other hand, current robots control their bodies by learning the network structure described by humans from their experiences, making certain assumptions on the relationship between sensors and actuators. In addition, the network model does not adapt to changes in the robot's body, the tools that are grasped, or the environment, and there is no unified theory, not only for control but also for state estimation, anomaly detection, simulation, and so on. In this study, we propose a Generalized Multisensory Correlational Model (GeMuCo), in which the robot itself acquires a body schema describing the correlation between sensors and actuators from its own experience, including model structures such as network input/output. The robot adapts to the current environment by updating this body schema model online, estimates and controls its body state, and even performs anomaly detection and simulation. We demonstrate the effectiveness of this method by applying it to tool-use considering changes in grasping state for an axis-driven robot, to joint-muscle mapping learning for a musculoskeletal robot, and to full-body tool manipulation for a low-rigidity plastic-made humanoid.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 33 references that are in the map. First posted on arXiv 10 Sep 2024.
Builds on
13 of 33 references.
References · 13
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
- 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
- 2017A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencodercited by 209
- 2014Robots that can adapt like animalscited by 204
- 2015Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Controlcited by 33
- 2019Multimodal representation models for prediction and control from partial informationcited by 11
- 2024Adaptive Whole-body Robotic Tool-use Learning on Low-rigidity Plastic-made Humanoids Using Vision and Tactile Sensorscited by 2
- 2024Online Learning of Joint-Muscle Mapping Using Vision in Tendon-driven Musculoskeletal Humanoidscited by 1
- 2024Deep Predictive Model Learning with Parametric Bias: Handling Modeling Difficulties and Temporal Model Changescited by 1
- 2024Musculoskeletal AutoEncoder: A Unified Online Acquisition Method of Intersensory Networks for State Estimation, Control, and Simulation of Musculoskeletal Humanoidscited by 1
Built on by
Not cited yet.