mlsanityData to 7 Oct 2026

Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models

arXiv 2102.11394 ↗First posted 22 Feb 2021cs.LG cs.RO cs.SYcited by 6

Not on the map. Why?

Abstract (arXiv)

In this paper, we learn dynamics models for parametrized families of dynamical systems with varying properties. The dynamics models are formulated as stochastic processes conditioned on a latent context variable which is inferred from observed transitions of the respective system. The probabilistic formulation allows us to compute an action sequence which, for a limited number of environment interactions, optimally explores the given system within the parametrized family. This is achieved by steering the system through transitions being most informative for the context variable. We demonstrate the effectiveness of our method for exploration on a non-linear toy-problem and two well-known reinforcement learning environments.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 22 Feb 2021.

Builds on

22 of 34 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2016OpenAI Gymcited by 1,684
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2017Deep Setscited by 1,394
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  9. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  10. 2018Conditional Neural Processescited by 421
  11. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  12. 2018Neural Processescited by 307
10 more
  1. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  2. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  3. 2019Variational Bayesian Optimal Experimental Designcited by 116
  4. 2017A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learningcited by 107
  5. 2018Model-Based Active Explorationcited by 105
  6. 2019Sequential Neural Processescited by 55
  7. 2020Reinforcement Learning through Active Inferencecited by 47
  8. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  9. 2019Actively Learning Gaussian Process Dynamicscited by 26
  10. 2020Neural Physicist: Learning Physical Dynamics from Image Sequencescited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 2.

6 papers.

  1. 2021An Experimental Design Perspective on Model-Based Reinforcement Learningcited by 16
  2. 2022Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarioscited by 8
  3. 2022Exploration via Planning for Information about the Optimal Trajectorycited by 8
  4. 2023Context-Conditional Navigation with a Learning-Based Terrain- and Robot-Aware Dynamics Modelcited by 1
  5. 2024Safe reinforcement learning in uncertain contextscited by 1
  6. 2024Learning a Terrain- and Robot-Aware Dynamics Model for Autonomous Mobile Robot Navigationcited by 1