mlsanityData to 14 Oct 2026

Hyperbolic Deep Reinforcement Learning

arXiv 2210.01542 ↗First posted 4 Oct 2022cs.LG cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

We propose a new class of deep reinforcement learning (RL) algorithms that model latent representations in hyperbolic space. Sequential decision-making requires reasoning about the possible future consequences of current behavior. Consequently, capturing the relationship between key evolving features for a given task is conducive to recovering effective policies. To this end, hyperbolic geometry provides deep RL models with a natural basis to precisely encode this inherently hierarchical information. However, applying existing methodologies from the hyperbolic deep learning literature leads to fatal optimization instabilities due to the non-stationarity and variance characterizing RL gradient estimators. Hence, we design a new general method that counteracts such optimization challenges and enables stable end-to-end learning with deep hyperbolic representations. We empirically validate our framework by applying it to popular on-policy and off-policy RL algorithms on the Procgen and Atari 100K benchmarks, attaining near universal performance and generalization benefits. Given its natural fit, we hope future RL research will consider hyperbolic representations as a standard tool.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 63 of its 97 references that are in the map. First posted on arXiv 4 Oct 2022.

Builds on

63 of 97 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2016Improved Techniques for Training GANscited by 2,970
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  8. 2017Improved Training of Wasserstein GANscited by 2,235
  9. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  11. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  12. 2015Prioritized Experience Replaycited by 1,099
38 more
  1. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  4. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  5. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  7. 2019Solving Rubik's Cube with a Robot Handcited by 734
  8. 2017Poincar'e Embeddings for Learning Hierarchical Representationscited by 684
  9. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  10. 2019Model-Based Reinforcement Learning for Ataricited by 502
  11. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  12. 2020Reinforcement Learning with Augmented Datacited by 468
  13. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 464
  14. 2018Hyperbolic Neural Networkscited by 402
  15. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  16. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  17. 2019Challenges of Real-World Reinforcement Learningcited by 291
  18. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  19. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  20. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  21. 2017Noisy Networks for Explorationcited by 266
  22. 2019Hyperbolic Image Embeddingscited by 221
  23. 2020Prioritized Level Replaycited by 198
  24. 2018Riemannian Adaptive Optimization Methodscited by 194
  25. 2018Poincar'e GloVe: Hyperbolic Word Embeddingscited by 156
  26. 2021Hyperbolic Deep Neural Networks: A Surveycited by 139
  27. 2020Phasic Policy Gradientcited by 123
  28. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 122
  29. 2019When to use parametric models in reinforcement learning?cited by 118
  30. 2019Continuous Hierarchical Representations with Poincar'e Variational Auto-Encoderscited by 112
  31. 2020Invariant Causal Prediction for Block MDPscited by 105
  32. 2017Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Controlcited by 93
  33. 2019Observational Overfitting in Reinforcement Learningcited by 91
  34. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  35. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  36. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  37. 2021Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapescited by 71
  38. 2018Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselinescited by 68

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 8; 2025: 6; 2026: 4.

25 papers.

Sort:

By area:

  1. 2023Hyperbolic Deep Learning in Computer Vision: A Surveycited by 73
  2. 2023EDGI: Equivariant Diffusion for Planning with Embodied Agentscited by 36
  3. 2024Hyperbolic Fine-Tuning for Large Language Modelscited by 27
  4. 2023Hyperbolic Diffusion Embedding and Distance for Hierarchical Representation Learningcited by 18
  5. 2022Hyperbolic Sliced-Wasserstein via Geodesic and Horospherical Projectionscited by 16
  6. 2024Embedding Geometries of Contrastive Language-Image Pre-Trainingcited by 6
  7. 2026GeoWorld: Geometric World Modelscited by 6
  8. 2023Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Designcited by 5
  9. 2026Stable Deep Reinforcement Learning via Isotropic Gaussian Representationscited by 5
  10. 2023Capacity Bounds for Hyperbolic Neural Network Representations of Latent Tree Structurescited by 4
  11. 2024Adversarial Attacks on Hyperbolic Networkscited by 3
  12. 2025A group-theoretic framework for machine learning in hyperbolic spacescited by 3
13 more
  1. 2024Why are hyperbolic neural networks effective? A study on hierarchical representation capabilitycited by 1
  2. 2024Kuramoto Oscillators and Swarms on Manifolds for Geometry Informed Machine Learningcited by 1
  3. 2024Hyp2Nav: Hyperbolic Planning and Curiosity for Crowd Navigationcited by 1
  4. 2025Hyperbolic Residual Quantization: Discrete Representations for Data with Latent Hierarchiescited by 1
  5. 2025Surrogate-Assisted Evolutionary Reinforcement Learning Based on Autoencoder and Hyperbolic Neural Networkcited by 1
  6. 2025Randomized HyperSteiner: A Stochastic Delaunay Triangulation Heuristic for the Hyperbolic Steiner Minimal Treecited by 1
  7. 2023HLoOP -- Hyperbolic 2-space Local Outlier Probabilitiescited by 0
  8. 2024A Bottom-Up Approach to Class-Agnostic Image Segmentationcited by 0
  9. 2024Uncovering RL Integration in SSL Loss: Objective-Specific Implications for Data-Efficient RLcited by 0
  10. 2025Free Random Projection for In-Context Reinforcement Learningcited by 0
  11. 2025Improved Offline Reinforcement Learning via Quantum Metric Encodingcited by 0
  12. 2026Curvature-Aware Captioning:Leveraging Geodesic Attention for 3D Scene Understandingcited by 0
  13. 2026Information-Geometric Optimization on Spherescited by 0