mlsanityData to 7 Oct 2026

Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems

arXiv 2411.04098 ↗First posted 6 Nov 2024cs.LG cs.CE math.OCcited by 2

Not on the map. Why?

Abstract (arXiv)

Effectively controlling systems governed by Partial Differential Equations (PDEs) is crucial in several fields of Applied Sciences and Engineering. These systems usually yield significant challenges to conventional control schemes due to their nonlinear dynamics, partial observability, high-dimensionality once discretized, distributed nature, and the requirement for low-latency feedback control. Reinforcement Learning (RL), particularly Deep RL (DRL), has recently emerged as a promising control paradigm for such systems, demonstrating exceptional capabilities in managing high-dimensional, nonlinear dynamics. However, DRL faces challenges including sample inefficiency, robustness issues, and an overall lack of interpretability. To address these issues, we propose a data-efficient, interpretable, and scalable Dyna-style Model-Based RL framework for PDE control, combining the Sparse Identification of Nonlinear Dynamics with Control (SINDy-C) algorithm and an autoencoder (AE) framework for the sake of dimensionality reduction of PDE states and actions. This novel approach enables fast rollouts, reducing the need for extensive environment interactions, and provides an interpretable latent space representation of the PDE forward dynamics. We validate our method on two PDE problems describing fluid flows - namely, the 1D Burgers equation and 2D Navier-Stokes equations - comparing it against a model-free baseline, and carrying out an extensive analysis of the learned dynamics.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 70 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

36 of 70 references.

References · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  6. 2020Mastering Atari with Discrete World Modelscited by 990
  7. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  8. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  10. 2018Learning Dexterous In-Hand Manipulationcited by 643
  11. 2019Machine Learning for Fluid Mechanicscited by 380
  12. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
24 more
  1. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  2. 2017Sparse identification of nonlinear dynamics for model predictive control in the low-data limitcited by 139
  3. 2021Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and controlcited by 134
  4. 2018State Representation Learning for Control: An Overviewcited by 125
  5. 2017A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problemcited by 123
  6. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 99
  7. 2021PySINDy: A comprehensive Python package for robust sparse system identificationcited by 88
  8. 2017Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Drivingcited by 81
  9. 2022Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoderscited by 58
  10. 2017RLlib: Abstractions for Distributed Reinforcement Learningcited by 53
  11. 2022Reduced order modeling of parametrized systems through autoencoders and SINDy approach: continuation of periodic solutionscited by 29
  12. 2022Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constantscited by 24
  13. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
  14. 2023Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learningcited by 21
  15. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  16. 2021SINDy with Control: A Tutorialcited by 15
  17. 2024PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equationscited by 8
  18. 2024VENI, VINDy, VICI: a generative reduced-order modeling framework with uncertainty quantificationcited by 8
  19. 2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policiescited by 7
  20. 2023Learning a model is paramount for sample efficiency in reinforcement learning control of PDEscited by 5
  21. 2018Feedback control of nonlinear PDEs using data-efficient reduced order models based on the Koopman operatorcited by 4
  22. 2022Model-Based Reinforcement Learning with SINDycited by 4
  23. 2023Online identification and control of PDEs via Reinforcement Learning methodscited by 3
  24. 2020Deep reinforcement learning for feedback control in a collective flashing ratchetcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025On the definition and importance of interpretability in scientific machine learningcited by 5
  2. 2026Global Optimality for Constrained Exploration via Penalty Regularizationcited by 0