mlsanityData to 13 Oct 2026

HyperPPO: A scalable method for finding small policies for robotic control

arXiv 2309.16663 ↗First posted 28 Sep 2023cs.RO cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Models with fewer parameters are necessary for the neural control of memory-limited, performant robots. Finding these smaller neural network architectures can be time-consuming. We propose HyperPPO, an on-policy reinforcement learning algorithm that utilizes graph hypernetworks to estimate the weights of multiple neural architectures simultaneously. Our method estimates weights for networks that are much smaller than those in common-use networks yet encode highly performant policies. We obtain multiple trained policies at the same time while maintaining sample efficiency and provide the user the choice of picking a network architecture that satisfies their computational constraints. We show that our method scales well - more training resources produce faster convergence to higher-performing architectures. We demonstrate that the neural policies estimated by HyperPPO are capable of decentralized control of a Crazyflie2.1 quadrotor. Website: https://sites.google.com/usc.edu/hyperppo

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Builds on 36 of its 43 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

36 of 43 references.

References · 36

  1. 2017Proximal Policy Optimization Algorithmscited by 14,592
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,656
  3. 2015Continuous control with deep reinforcement learningcited by 3,290
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,789
  5. 2016OpenAI Gymcited by 1,694
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,496
  7. 2017Hindsight Experience Replaycited by 1,104
  8. 2016HyperNetworkscited by 1,010
  9. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 890
  10. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 399
  11. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 379
  12. 2019Continual learning with hypernetworkscited by 268
24 more
  1. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 201
  2. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 161
  3. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  4. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  5. 2018Graph HyperNetworks for Neural Architecture Searchcited by 131
  6. 2021Learning High-Speed Flight in the Wildcited by 98
  7. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  8. 2020Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learningcited by 85
  9. 2017Control of a Quadrotor with Reinforcement Learningcited by 75
  10. 2022EnvPool: A Highly Parallel Reinforcement Learning Environment Execution Enginecited by 60
  11. 2021Autonomous Drone Racing with Deep Reinforcement Learningcited by 51
  12. 2019Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learningcited by 40
  13. 2019Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotorscited by 36
  14. 2023Hyper-Decision Transformer for Efficient Online Policy Adaptationcited by 32
  15. 2023Proximal Policy Gradient Arborescence for Quality Diversity Reinforcement Learningcited by 20
  16. 2021HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworkscited by 18
  17. 2021Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learningcited by 18
  18. 2023Generating Behaviorally Diverse Policies with Latent Diffusion Modelscited by 18
  19. 2023Collision Avoidance and Navigation for a Quadrotor Swarm Using End-to-end Deep Reinforcement Learningcited by 6
  20. 2023QuadSwarm: A Modular Multi-Quadrotor Simulator for Deep Reinforcement Learning with Direct Thrust Controlcited by 5
  21. 2020Optimizing the Neural Architecture of Reinforcement Learning Agentscited by 4
  22. 2021Visionary: Vision architecture discovery for robot learningcited by 4
  23. 2021Differentiable Architecture Search for Reinforcement Learningcited by 4
  24. 2022Efficiently Learning Small Policies for Locomotion and Manipulationcited by 4

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2025Hyper-GoalNet: Goal-Conditioned Manipulation Policy Learning with HyperNetworkscited by 4
  2. 2024WARPD: World model Assisted Reactive Policy Diffusioncited by 3
  3. 2026Benefits of Low-Cost Bio-Inspiration in the Age of Overparametrizationcited by 0
  4. 2026E2DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulationcited by 0