Situationally-Aware Dynamics Learning
Not on the map. Why?
Abstract (arXiv)
Autonomous robots operating in complex, unstructured environments face significant challenges due to latent, unobserved factors that obscure their understanding of both their internal state and the external world. Addressing this challenge would enable robots to develop a more profound grasp of their operational context. To tackle this, we propose a novel framework for online learning of hidden state representations, with which the robots can adapt in real-time to uncertain and dynamic conditions that would otherwise be ambiguous and result in suboptimal or erroneous behaviors. Our approach is formalized as a Generalized Hidden Parameter Markov Decision Process, which explicitly models the influence of unobserved parameters on both transition dynamics and reward structures. Our core innovation lies in learning online the joint distribution of state transitions, which serves as an expressive representation of latent ego- and environmental-factors. This probabilistic approach supports the identification and adaptation to different operational situations, improving robustness and safety. Through a multivariate extension of Bayesian Online Changepoint Detection, our method segments changes in the underlying data generating process governing the robot's dynamics. The robot's transition model is then informed with a symbolic representation of the current situation derived from the joint distribution of latest state transitions, enabling adaptive and context-aware decision-making. To showcase the real-world effectiveness, we validate our approach in the challenging task of unstructured terrain navigation, where unmodeled and unmeasured terrain characteristics can significantly impact the robot's motion. Extensive experiments in both simulation and real world reveal significant improvements in data efficiency, policy performance, and the emergence of safer, adaptive navigation strategies.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 53 of its 117 references that are in the map. First posted on arXiv 26 May 2025.
Builds on
53 of 117 references.
References · 53
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
- 2018World Modelscited by 1,513
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
- 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
- 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
38 more
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
- 2015Contextual Markov Decision Processescited by 220
- 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
- 2020Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learningcited by 98
- 2013Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizationscited by 70
- 2021RL for Latent MDPs: Regret Guarantees and a Lower Boundcited by 70
- 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 65
- 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 58
- 2022Adapting Rapid Motor Adaptation for Bipedal Robotscited by 54
- 2016Self-Correcting Models for Model-Based Reinforcement Learningcited by 43
- 2021MBRL-Lib: A Modular Library for Model-based Reinforcement Learningcited by 42
- 2020Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learningcited by 31
- 2017Robust and Efficient Transfer Learning with Hidden-Parameter Markov Decision Processescited by 29
- 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulationcited by 28
- 2022How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversabilitycited by 27
- 2020Generalized Hidden Parameter MDPs Transferable Model-based RL in a Handful of Trialscited by 20
- 2022Risk-Aware Off-Road Navigation via a Learned Speed Distribution Mapcited by 20
- 2021Robust Model Predictive Path Integral Control: Analysis and Performance Guaranteescited by 17
- 2022Learning Efficient Abstract Planning Models that Choose What to Predictcited by 17
- 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
- 2023Rapid Motor Adaptation for Robotic Manipulator Armscited by 14
- 2022Probabilistic Traversability Model for Risk-Aware Motion Planning in Off-Road Environmentscited by 13
- 2022Physics Embedded Neural Network Vehicle Model and Applications in Risk-Aware Autonomous Driving Using Latent Featurescited by 12
- 2024RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashescited by 10
- 2016Changepoint Detection in the Presence of Outlierscited by 9
- 2023PyFlyt -- UAV Simulation Environments for Reinforcement Learning Researchcited by 9
- 2021Talk-to-Resolve: Combining scene understanding and spatial dialogue to resolve granular task ambiguity for a collocated robotcited by 7
- 2024PhysORD: A Neuro-Symbolic Approach for Physics-infused Motion Prediction in Off-road Drivingcited by 7
- 2024Adaptive Diffusion Terrain Generator for Autonomous Uneven Terrain Navigationcited by 6
- 2019Learning Latent State Spaces for Planning through Reward Predictioncited by 4
- 2024Fast Extrinsic Calibration for Multiple Inertial Measurement Units in Visual-Inertial Systemcited by 4
- 2021Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrainscited by 3
- 2022Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planningcited by 3
- 2022RAMP: A Risk-Aware Mapping and Planning Pipeline for Fast Off-Road Ground Robot Navigationcited by 3
- 2024Autonomous Mapless Navigation on Uneven Terrainscited by 3
- 2024Gaussian Process-based Traversability Analysis for Terrain Mapless Navigationcited by 3
- 2024Traversability-aware Adaptive Optimization for Path Planning and Control in Mountainous Terraincited by 3
- 2021Latent Representation in Human-Robot Interaction with Explicit Consideration of Periodic Dynamicscited by 2
50 most cited of 53 All 53 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.