# Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers

arXiv: https://arxiv.org/abs/2107.11472 · HTML: https://mlsanity.com/paper/2107.11472/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 23 Jul 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Energy-based Out-of-distribution Detection](https://mlsanity.com/paper/2010.03759/) (2020)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry](https://mlsanity.com/paper/1806.03417/) (2018)
- [Hyperbolic Graph Neural Networks](https://mlsanity.com/paper/1910.12892/) (2019)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking](https://mlsanity.com/paper/1504.06755/) (2015)
- [Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice](https://mlsanity.com/paper/1711.04735/) (2017)
- [All you need is a good init](https://mlsanity.com/paper/1511.06422/) (2015)
- [Hyperbolic Attention Networks](https://mlsanity.com/paper/1805.09786/) (2018)
- [Continuous Hierarchical Representations with Poincar'e Variational Auto-Encoders](https://mlsanity.com/paper/1901.06033/) (2019)
- [Which Neural Net Architectures Give Rise To Exploding and Vanishing Gradients?](https://mlsanity.com/paper/1801.03744/) (2018)
- [Beyond Convexity: Stochastic Quasi-Convex Optimization](https://mlsanity.com/paper/1507.02030/) (2015)
- [The Emergence of Spectral Universality in Deep Networks](https://mlsanity.com/paper/1802.09979/) (2018)
- [Hyperbolic Graph Attention Network](https://mlsanity.com/paper/1912.03046/) (2019)
- [Large-Margin Classification in Hyperbolic Space](https://mlsanity.com/paper/1806.00437/) (2018)
- [Robust Large-Margin Learning in Hyperbolic Space](https://mlsanity.com/paper/2004.05465/) (2020)

## Built on by (most cited first, 30 of 57)

- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [Fully Hyperbolic Convolutional Neural Networks for Computer Vision](https://mlsanity.com/paper/2303.15919/) (2023)
- [The Numerical Stability of Hyperbolic Representation Learning](https://mlsanity.com/paper/2211.00181/) (2022)
- [HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representations](https://mlsanity.com/paper/2303.06242/) (2023)
- [Hyperbolic Deep Reinforcement Learning](https://mlsanity.com/paper/2210.01542/) (2022)
- [Poincar'e ResNet](https://mlsanity.com/paper/2303.14027/) (2023)
- [Hyperbolic Category Discovery](https://mlsanity.com/paper/2504.06120/) (2025)
- [Hyperbolic Active Learning for Semantic Segmentation under Domain Shift](https://mlsanity.com/paper/2306.11180/) (2023)
- [Learning Structured Representations with Hyperbolic Embeddings](https://mlsanity.com/paper/2412.01023/) (2024)
- [Hyperbolic Contrastive Learning](https://mlsanity.com/paper/2302.01409/) (2023)
- [Hyperbolic Learning with Synthetic Captions for Open-World Detection](https://mlsanity.com/paper/2404.05016/) (2024)
- [HyperCore: The Core Framework for Building Hyperbolic Foundation Models with Comprehensive Modules](https://mlsanity.com/paper/2504.08912/) (2025)
- [Understanding Hyperbolic Metric Learning through Hard Negative Sampling](https://mlsanity.com/paper/2404.15523/) (2024)
- [Hyperbolic Learning with Multimodal Large Language Models](https://mlsanity.com/paper/2408.05097/) (2024)
- [Flattening the Parent Bias: Hierarchical Semantic Segmentation in the Poincar'e Ball](https://mlsanity.com/paper/2404.03778/) (2024)
- [Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval](https://mlsanity.com/paper/2411.17490/) (2024)
- [Equiangular Basis Vectors](https://mlsanity.com/paper/2303.11637/) (2023)
- [A Fashion Item Recommendation Model in Hyperbolic Space](https://mlsanity.com/paper/2409.02599/) (2024)
- [HMSN: Hyperbolic Self-Supervised Learning by Clustering with Ideal Prototypes](https://mlsanity.com/paper/2305.10926/) (2023)
- [Hyperbolic vs Euclidean Embeddings in Few-Shot Learning: Two Sides of the Same Coin](https://mlsanity.com/paper/2309.10013/) (2023)
- [Neural Networks on Symmetric Spaces of Noncompact Type](https://mlsanity.com/paper/2601.01097/) (2026)
- [Continual Hyperbolic Learning of Instances and Classes](https://mlsanity.com/paper/2506.10710/) (2025)
- [Hyperbolic Busemann Neural Networks](https://mlsanity.com/paper/2602.18858/) (2026)
- [Hyperbolic Geometry in Computer Vision: A Survey](https://mlsanity.com/paper/2304.10764/) (2023)
- [When Cars meet Drones: Hyperbolic Federated Learning for Source-Free Domain Adaptation in Adverse Weather](https://mlsanity.com/paper/2403.13762/) (2024)
- [Robust Hyperbolic Learning with Curvature-Aware Optimization](https://mlsanity.com/paper/2405.13979/) (2024)
- [Curvature Learning for Generalization of Hyperbolic Neural Networks](https://mlsanity.com/paper/2508.17232/) (2025)
- [Hyp-UML: Hyperbolic Image Retrieval with Uncertainty-aware Metric Learning](https://mlsanity.com/paper/2310.08390/) (2023)
- [Hyperbolic Metric Learning for Visual Outlier Detection](https://mlsanity.com/paper/2403.15260/) (2024)
- [Hyp-OC: Hyperbolic One Class Classification for Face Anti-Spoofing](https://mlsanity.com/paper/2404.14406/) (2024)

## Abstract (arXiv)

Hyperbolic space can naturally embed hierarchies, unlike Euclidean space. Hyperbolic Neural Networks (HNNs) exploit such representational power by lifting Euclidean features into hyperbolic space for classification, outperforming Euclidean neural networks (ENNs) on datasets with known semantic hierarchies. However, HNNs underperform ENNs on standard benchmarks without clear hierarchies, greatly restricting HNNs' applicability in practice. Our key insight is that HNNs' poorer general classification performance results from vanishing gradients during backpropagation, caused by their hybrid architecture connecting Euclidean features to a hyperbolic classifier. We propose an effective solution by simply clipping the Euclidean feature magnitude while training HNNs. Our experiments demonstrate that clipped HNNs become super-hyperbolic classifiers: They are not only consistently better than HNNs which already outperform ENNs on hierarchical data, but also on-par with ENNs on MNIST, CIFAR10, CIFAR100 and ImageNet benchmarks, with better adversarial robustness and out-of-distribution detection.