mlsanityData to 14 Oct 2026

ODE-based Recurrent Model-free Reinforcement Learning for POMDPs

arXiv 2309.14078 ↗First posted 25 Sep 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Neural ordinary differential equations (ODEs) are widely recognized as the standard for modeling physical mechanisms, which help to perform approximate inference in unknown physical or biological environments. In partially observable (PO) environments, how to infer unseen information from raw observations puzzled the agents. By using a recurrent policy with a compact context, context-based reinforcement learning provides a flexible way to extract unobservable information from historical transitions. To help the agent extract more dynamics-related information, we present a novel ODE-based recurrent model combines with model-free reinforcement learning (RL) framework to solve partially observable Markov decision processes (POMDPs). We experimentally demonstrate the efficacy of our methods across various PO continuous control and meta-RL tasks. Furthermore, our experiments illustrate that our method is robust against irregular observations, owing to the ability of ODEs to model irregularly-sampled time series.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

36 of 52 references.

References · 36

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2018Neural Ordinary Differential Equationscited by 3,779
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  8. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  10. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
  11. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  12. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
24 more
  1. 2022Kubric: A scalable dataset generatorcited by 381
  2. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 366
  3. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  4. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 231
  5. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  6. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
  7. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 183
  8. 2020Liquid Time-constant Networkscited by 161
  9. 2018Meta-Gradient Reinforcement Learningcited by 159
  10. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  11. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 122
  12. 2020Learning Long-Term Dependencies in Irregularly-Sampled Time Seriescited by 83
  13. 2015Memory-based control with recurrent neural networkscited by 78
  14. 2017Recurrent Environment Simulatorscited by 68
  15. 2021Memory-based Deep Reinforcement Learning for POMDPscited by 54
  16. 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 47
  17. 2021Continuous-Time Model-Based Reinforcement Learningcited by 41
  18. 2020Causal models for dynamical systemscited by 34
  19. 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
  20. 2021Deep Molecular Representation Learning via Fusing Physical and Chemical Informationcited by 21
  21. 2023Complex Dynamic Neurons Improved Spiking Transformer Network for Efficient Automatic Speech Recognitioncited by 20
  22. 2022A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learningcited by 15
  23. 2020Faster Policy Learning with Continuous-Time Gradientscited by 8
  24. 2022Tuning Synaptic Connections instead of Weights by Genetic Algorithm in Spiking Policy Networkcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2; 2025: 4.

7 papers.

  1. 2025Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domainscited by 7
  2. 2024Biologically-Plausible Topology Improved Spiking Actor Network for Efficient Deep Reinforcement Learningcited by 2
  3. 2023Enhancing Solutions for Complex PDEs: Introducing Complementary Convolution and Equivariant Attention in Fourier Neural Operatorscited by 1
  4. 2024Multiscale fusion enhanced spiking neural network for invasive BCI neural signal decodingcited by 0
  5. 2025A Convolution and Attention Based Encoder for Reinforcement Learning under Partial Observabilitycited by 0
  6. 2025Success in Humanoid Reinforcement Learning under Partial Observationcited by 0
  7. 2025Learning Causal States Under Partial Observability and Perturbationcited by 0