# Transfer Learning for a Class of Cascade Dynamical Systems

arXiv: https://arxiv.org/abs/2410.06828 · HTML: https://mlsanity.com/paper/2410.06828/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 9 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Sim-to-Real Robot Learning from Pixels with Progressive Nets](https://mlsanity.com/paper/1610.04286/) (2016)
- [Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control](https://mlsanity.com/paper/1706.06491/) (2017)
- [Simulation-based reinforcement learning for real-world autonomous driving](https://mlsanity.com/paper/1911.12905/) (2019)
- [Data-efficient Domain Randomization with Bayesian Optimization](https://mlsanity.com/paper/2003.02471/) (2020)
- [Probabilistic Constraint for Safety-Critical Reinforcement Learning](https://mlsanity.com/paper/2306.17279/) (2023)
- [Reinforcement Learning for Reduced-order Models of Legged Robots](https://mlsanity.com/paper/2310.09873/) (2023)
- [Deep Residual Reinforcement Learning based Autonomous Blimp Control](https://mlsanity.com/paper/2203.05360/) (2022)
- [Input-to-State Stability in Probability](https://mlsanity.com/paper/2304.14578/) (2023)
- [Blackout Mitigation via Physics-guided RL](https://mlsanity.com/paper/2401.09640/) (2024)
- [Contraction Analysis of Discrete-time Stochastic Systems](https://mlsanity.com/paper/2106.05635/) (2021)
- [Beyond Inverted Pendulums: Task-optimal Simple Models of Legged Locomotion](https://mlsanity.com/paper/2301.02075/) (2023)

## Built on by (most cited first, 1 of 1)

- [Safety Guarantees in Zero-Shot Reinforcement Learning for Cascade Dynamical Systems](https://mlsanity.com/paper/2604.10429/) (2026)

## Abstract (arXiv)

This work considers the problem of transfer learning in the context of reinforcement learning. Specifically, we consider training a policy in a reduced order system and deploying it in the full state system. The motivation for this training strategy is that running simulations in the full-state system may take excessive time if the dynamics are complex. While transfer learning alleviates the computational issue, the transfer guarantees depend on the discrepancy between the two systems. In this work, we consider a class of cascade dynamical systems, where the dynamics of a subset of the state-space influence the rest of the states but not vice-versa. The reinforcement learning policy learns in a model that ignores the dynamics of these states and treats them as commanded inputs. In the full-state system, these dynamics are handled using a classic controller (e.g., a PID). These systems have vast applications in the control literature and their structure allows us to provide transfer guarantees that depend on the stability of the inner loop controller. Numerical experiments on a quadrotor support the theoretical findings.