# Fast Online Adaptation in Robotics through Meta-Learning Embeddings of Simulated Priors

arXiv: https://arxiv.org/abs/2003.04663 · HTML: https://mlsanity.com/paper/2003.04663/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2020.

## Built on by (most cited first, 22 of 22)

- [PointVLA: Injecting the 3D World into Vision-Language-Action Models](https://mlsanity.com/paper/2503.07511/) (2025)
- [Context is Everything: Implicit Identification for Dynamics Adaptation](https://mlsanity.com/paper/2203.05549/) (2022)
- [Meta-learning approaches for few-shot learning: A survey of recent advances](https://mlsanity.com/paper/2303.07502/) (2023)
- [Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires](https://mlsanity.com/paper/2109.08522/) (2021)
- [ObjectVLA: End-to-End Open-World Object Manipulation Without Demonstration](https://mlsanity.com/paper/2502.19250/) (2025)
- [Transformers are Adaptable Task Planners](https://mlsanity.com/paper/2207.02442/) (2022)
- [Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control](https://mlsanity.com/paper/2210.12583/) (2022)
- [Federated Learning and Meta Learning: Approaches, Applications, and Directions](https://mlsanity.com/paper/2210.13111/) (2022)
- [Robust Reinforcement Learning via Genetic Curriculum](https://mlsanity.com/paper/2202.08393/) (2022)
- [Meta-Reinforcement Learning by Tracking Task Non-stationarity](https://mlsanity.com/paper/2105.08834/) (2021)
- [Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters](https://mlsanity.com/paper/2503.18684/) (2025)
- [MPC of Uncertain Nonlinear Systems with Meta-Learning for Fast Adaptation of Neural Predictive Models](https://mlsanity.com/paper/2404.12097/) (2024)
- [Graph Neural Network Expressivity and Meta-Learning for Molecular Property Regression](https://mlsanity.com/paper/2209.13410/) (2022)
- [Task-agnostic Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation](https://mlsanity.com/paper/2410.02995/) (2024)
- [CARoL: Context-aware Adaptation for Robot Learning](https://mlsanity.com/paper/2506.07006/) (2025)
- [Meta-Reinforcement Learning for Adaptive Motor Control in Changing Robot Dynamics and Environments](https://mlsanity.com/paper/2101.07599/) (2021)
- [First do not fall: learning to exploit a wall with a damaged humanoid robot](https://mlsanity.com/paper/2203.00316/) (2022)
- [RL3: Boosting Meta Reinforcement Learning via RL inside RL2](https://mlsanity.com/paper/2306.15909/) (2023)
- [Hypothesis Network Planned Exploration for Rapid Meta-Reinforcement Learning Adaptation](https://mlsanity.com/paper/2311.03701/) (2023)
- [Evaluating Pretrained models for Deployable Lifelong Learning](https://mlsanity.com/paper/2311.13648/) (2023)
- [Real-Time Performance Analysis of Multi-Fidelity Residual Physics-Informed Neural Process-Based State Estimation for Robotic Systems](https://mlsanity.com/paper/2511.08231/) (2025)
- [In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation](https://mlsanity.com/paper/2610.13411/) (2026)

## Abstract (arXiv)

Meta-learning algorithms can accelerate the model-based reinforcement learning (MBRL) algorithms by finding an initial set of parameters for the dynamical model such that the model can be trained to match the actual dynamics of the system with only a few data-points. However, in the real world, a robot might encounter any situation starting from motor failures to finding itself in a rocky terrain where the dynamics of the robot can be significantly different from one another. In this paper, first, we show that when meta-training situations (the prior situations) have such diverse dynamics, using a single set of meta-trained parameters as a starting point still requires a large number of observations from the real system to learn a useful model of the dynamics. Second, we propose an algorithm called FAMLE that mitigates this limitation by meta-training several initial starting points (i.e., initial parameters) for training the model and allows the robot to select the most suitable starting point to adapt the model to the current situation with only a few gradient steps. We compare FAMLE to MBRL, MBRL with a meta-trained model with MAML, and model-free policy search algorithm PPO for various simulated and real robotic tasks, and show that FAMLE allows the robots to adapt to novel damages in significantly fewer time-steps than the baselines.