# Towards Generalization and Simplicity in Continuous Control

arXiv: https://arxiv.org/abs/1703.02660 · HTML: https://mlsanity.com/paper/1703.02660/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2017.

## Built on by (most cited first, 30 of 68)

- [MOReL : Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2005.05951/) (2020)
- [The Unsurprising Effectiveness of Pre-Trained Vision Models for Control](https://mlsanity.com/paper/2203.03580/) (2022)
- [The Surprising Effectiveness of Representation Learning for Visual Imitation](https://mlsanity.com/paper/2112.01511/) (2021)
- [Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.05265/) (2021)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Decoupling Value and Policy for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2102.10330/) (2021)
- [Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?](https://mlsanity.com/paper/2204.11134/) (2022)
- [XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2312.12044/) (2023)
- [MobILE: Model-Based Imitation Learning From Observation Alone](https://mlsanity.com/paper/2102.10769/) (2021)
- [SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies](https://mlsanity.com/paper/2106.09678/) (2021)
- [Instabilities of Offline RL with Pre-Trained Neural Representation](https://mlsanity.com/paper/2103.04947/) (2021)
- [Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications](https://mlsanity.com/paper/2408.10215/) (2024)
- [An Optimistic Perspective on Offline Reinforcement Learning](https://mlsanity.com/paper/1907.04543/) (2019)
- [On the Importance of Exploration for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2306.05483/) (2023)
- [Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments](https://mlsanity.com/paper/2101.08152/) (2021)
- [The Generalization Gap in Offline Reinforcement Learning](https://mlsanity.com/paper/2312.05742/) (2023)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Policy Gradient Methods Find the Nash Equilibrium in N-player General-sum Linear-quadratic Games](https://mlsanity.com/paper/2107.13090/) (2021)
- [Investigating the Properties of Neural Network Representations in Reinforcement Learning](https://mlsanity.com/paper/2203.15955/) (2022)
- [In-Context Reinforcement Learning for Variable Action Spaces](https://mlsanity.com/paper/2312.13327/) (2023)
- [Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updates](https://mlsanity.com/paper/2310.19807/) (2023)
- [When Is Generalizable Reinforcement Learning Tractable?](https://mlsanity.com/paper/2101.00300/) (2021)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning](https://mlsanity.com/paper/2103.12726/) (2021)
- [Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributions](https://mlsanity.com/paper/2205.10218/) (2022)
- [Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach](https://mlsanity.com/paper/2304.12502/) (2023)
- [Memory-Consistent Neural Networks for Imitation Learning](https://mlsanity.com/paper/2310.06171/) (2023)
- [REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer](https://mlsanity.com/paper/2202.05244/) (2022)
- [Improving Generalization in Meta-RL with Imaginary Tasks from Latent Dynamics Mixture](https://mlsanity.com/paper/2105.13524/) (2021)
- [Locality Sensitive Sparse Encoding for Learning World Models Online](https://mlsanity.com/paper/2401.13034/) (2024)

## Abstract (arXiv)

This work shows that policies with simple linear and RBF parameterizations can be trained to solve a variety of continuous control tasks, including the OpenAI gym benchmarks. The performance of these trained policies are competitive with state of the art results, obtained with more elaborate parameterizations such as fully connected neural networks. Furthermore, existing training and testing scenarios are shown to be very limited and prone to over-fitting, thus giving rise to only trajectory-centric policies. Training with a diverse initial state distribution is shown to produce more global policies with better generalization. This allows for interactive control scenarios where the system recovers from large on-line perturbations; as shown in the supplementary video.