mlsanityData to 7 Oct 2026

Koopman Spectrum Nonlinear Regulators and Efficient Online Learning

arXiv 2106.15775 ↗First posted 30 Jun 2021cs.LG cs.RO cs.SYcited by 5

Not on the map. Why?

Abstract (arXiv)

Most modern reinforcement learning algorithms optimize a cumulative single-step cost along a trajectory. The optimized motions are often 'unnatural', representing, for example, behaviors with sudden accelerations that waste energy and lack predictability. In this work, we present a novel paradigm of controlling nonlinear systems via the minimization of the Koopman spectrum cost: a cost over the Koopman operator of the controlled dynamics. This induces a broader class of dynamical behaviors that evolve over stable manifolds such as nonlinear oscillators, closed loops, and smooth movements. We demonstrate that some dynamics characterizations that are not possible with a cumulative cost are feasible in this paradigm, which generalizes the classical eigenstructure and pole assignments to nonlinear decision making. Moreover, we present a sample efficient online learning algorithm for our problem that enjoys a sub-linear regret bound under some structural assumptions.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 25 of its 66 references that are in the map. First posted on arXiv 30 Jun 2021.

Builds on

25 of 66 references.

References · 25

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2018DeepMind Control Suitecited by 935
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  5. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 290
  6. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  7. 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 134
  8. 2020Naive Exploration is Optimal for Online LQRcited by 111
  9. 2020Learning Stable Deep Dynamics Modelscited by 110
  10. 2020Information Theoretic Regret Bounds for Online Nonlinear Controlcited by 87
  11. 2020Efficient Model-Based Reinforcement Learning through Optimistic Policy Search and Planningcited by 83
  12. 2020Active Learning for Nonlinear System Identification with Guaranteescited by 78
13 more
  1. 2019Learning Compositional Koopman Operators for Model-Based Controlcited by 67
  2. 2017Towards Generalization and Simplicity in Continuous Controlcited by 65
  3. 2017Data-driven discovery of Koopman eigenfunctions for controlcited by 64
  4. 2015Operator-valued Kernels for Learning from Functional Response Datacited by 59
  5. 2018Optimal construction of Koopman eigenfunctions for prediction and controlcited by 34
  6. 2019Active Learning of Dynamics for Data-Driven Control Using Koopman Operatorscited by 31
  7. 2016Random Fourier Features for Operator-Valued Kernelscited by 22
  8. 2020Learning Dynamics Models with Stable Invariant Setscited by 18
  9. 2020Learning Stable Models for Prediction and Controlcited by 11
  10. 2020Kernel Analog Forecasting: Multiscale Test Problemscited by 11
  11. 2020Neural Dynamic Mode Decomposition for End-to-End Modeling of Nonlinear Dynamicscited by 9
  12. 2018Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operatorscited by 7
  13. 2020Lyceum: An efficient and scalable ecosystem for robot learningcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 1.

5 papers.

  1. 2021Koopman Q-learning: Offline Reinforcement Learning via Symmetries of Dynamicscited by 27
  2. 2022Data-driven End-to-end Learning of Pole Placement Control for Nonlinear Dynamics via Koopman Invariant Subspacescited by 2
  3. 2022Dynamic Structure Estimation from Bandit Feedback using Nonvanishing Exponential Sumscited by 0
  4. 2022Modeling Nonlinear Dynamics in Continuous Time with Inductive Biases on Decay Rates and/or Frequenciescited by 0
  5. 2023Signatures Meet Dynamic Programming: Generalizing Bellman Equations for Trajectory Followingcited by 0