mlsanityData to 14 Oct 2026

Online Control-Informed Learning

arXiv 2410.03924 ↗First posted 4 Oct 2024math.OC cs.LG cs.ROcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper proposes an Online Control-Informed Learning (OCIL) framework, which employs the well-established optimal control and state estimation techniques in the field of control to solve a broad class of learning tasks in an online fashion. This novel integration effectively handles practical issues in machine learning such as noisy measurement data, online learning, and data efficiency. By considering any robot as a tunable optimal control system, we propose an online parameter estimator based on extended Kalman filter (EKF) to incrementally tune the system in an online fashion, enabling it to complete designated learning or control tasks. The proposed method also improves the robustness in learning by effectively managing noise in the data. Theoretical analysis is provided to demonstrate the convergence of OCIL. Three learning modes of OCIL, i.e. Online Imitation Learning, Online System Identification, and Policy Tuning On-the-fly, are investigated via experiments, which validate their effectiveness.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 56 of its 115 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

56 of 115 references.

References · 56

  1. 2018Neural Ordinary Differential Equationscited by 3,779
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2016End to End Learning for Self-Driving Carscited by 1,009
  5. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
  6. 2018Behavioral Cloning from Observationcited by 492
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  8. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  9. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 260
  10. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  11. 2018Task-Free Continual Learningcited by 221
  12. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 212
38 more
  1. 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 205
  2. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  3. 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 184
  4. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
  5. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  6. 2018Self-Imitation Learningcited by 138
  7. 2016Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Roboticscited by 134
  8. 2021KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamicscited by 132
  9. 2014Dynamic mode decomposition with controlcited by 129
  10. 2019Imitation Learning from Imperfect Demonstrationcited by 114
  11. 2023Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictorscited by 113
  12. 2019Distilling Policy Distillationcited by 84
  13. 2017Maximum Principle Based Algorithms for Deep Learningcited by 78
  14. 2018A Mean-Field Optimal Control Formulation of Deep Learningcited by 67
  15. 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61
  16. 2021Scalable Bayesian Inverse Reinforcement Learningcited by 56
  17. 2016Deep Learning Approximation for Stochastic Control Problemscited by 55
  18. 2016Generalizing Koopman Theory to allow for inputs and controlcited by 53
  19. 2019Pontryagin Differentiable Programming: An End-to-End Learning and Control Frameworkcited by 52
  20. 2023Deep networks for system identification: a Surveycited by 37
  21. 2019Variational Integrator Networks for Physically Structured Embeddingscited by 36
  22. 2017Second-Order Optimization for Non-Convex Machine Learning: An Empirical Studycited by 35
  23. 2019Deep learning as optimal control problems: models and numerical methodscited by 34
  24. 2019Active Learning of Dynamics for Data-Driven Control Using Koopman Operatorscited by 32
  25. 2018Universal Planning Networkscited by 30
  26. 2021Implicit energy regularization of neural ordinary-differential-equation controlcited by 29
  27. 2018Online Optimization with Predictions and Switching Costs: Fast Algorithms and the Fundamental Limitcited by 27
  28. 2018An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networkscited by 26
  29. 2021Safe Pontryagin Differentiable Programmingcited by 26
  30. 2018Composable Planning with Attributescited by 21
  31. 2017Path Integral Networks: End-to-End Differentiable Optimal Controlcited by 16
  32. 2020A Data-Driven Automatic Tuning Method for MPC under Uncertainty using Constrained Bayesian Optimizationcited by 16
  33. 2018MPC-Inspired Neural Network Policies for Sequential Decision Makingcited by 13
  34. 2022Deep Subspace Encoders for Nonlinear System Identificationcited by 13
  35. 2020Learning from Sparse Demonstrationscited by 9
  36. 2020Performance-Driven Cascade Controller Tuning with Bayesian Optimizationcited by 8
  37. 2019Selection dynamics for deep neural networkscited by 6
  38. 2020Learning from Human Directional Correctionscited by 6

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025DiffCoord: Differentiable Coordination for Distributed Multi-Agent Trajectory Optimizationcited by 2
  2. 2025Safe Online Control-Informed Learningcited by 1
  3. 2025A Control-Barrier-Function-Based Algorithm for Policy Adaptation in Reinforcement Learningcited by 0
  4. 2026Online Intention Prediction via Control-Informed Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Efficient Model-Based Reinforcement Learning for Robot Control via Online Optimization
  2. 2020Online inverse reinforcement learning with limited data
  3. 2025What Matters for Batch Online Reinforcement Learning in Robotics?
  4. 2018Online inverse reinforcement learning for nonlinear systems
  5. 2025Online Learning-Based Predictive Control for Nonlinear System ↗
  6. 2018A Framework and Method for Online Inverse Reinforcement Learning
  7. 2026A Review of Online Diffusion Policy RL Algorithms for Scalable Robotic Control
  8. 2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning