# Hyperbolic Contrastive Learning for Visual Representations beyond Objects

arXiv: https://arxiv.org/abs/2212.00653 · HTML: https://mlsanity.com/paper/2212.00653/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 79 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 30 of 53)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Hyperbolic Neural Networks](https://mlsanity.com/paper/1805.09112/) (2018)
- [Hyperbolic Graph Convolutional Neural Networks](https://mlsanity.com/paper/1910.12933/) (2019)
- [Are we done with ImageNet?](https://mlsanity.com/paper/2006.07159/) (2020)
- [Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2011.10043/) (2020)
- [Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry](https://mlsanity.com/paper/1806.03417/) (2018)
- [Hyperbolic Graph Neural Networks](https://mlsanity.com/paper/1910.12892/) (2019)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [Hyperbolic Image Embeddings](https://mlsanity.com/paper/1904.02239/) (2019)

## Built on by (most cited first, 30 of 61)

- [Hyperbolic Image-Text Representations](https://mlsanity.com/paper/2304.09172/) (2023)
- [Compositional Entailment Learning for Hyperbolic Vision-Language Models](https://mlsanity.com/paper/2410.06912/) (2024)
- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [HELM: Hyperbolic Large Language Models via Mixture-of-Curvature Experts](https://mlsanity.com/paper/2505.24722/) (2025)
- [Structuring Representation Geometry with Rotationally Equivariant Contrastive Learning](https://mlsanity.com/paper/2306.13924/) (2023)
- [Hyperbolic Deep Learning for Foundation Models: A Survey](https://mlsanity.com/paper/2507.17787/) (2025)
- [Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges](https://mlsanity.com/paper/2311.18044/) (2023)
- [Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries](https://mlsanity.com/paper/2504.08896/) (2025)
- [HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts](https://mlsanity.com/paper/2404.17507/) (2024)
- [Hyperbolic Safety-Aware Vision-Language Models](https://mlsanity.com/paper/2503.12127/) (2025)
- [Learning Structured Representations with Hyperbolic Embeddings](https://mlsanity.com/paper/2412.01023/) (2024)
- [Hyp-OW: Exploiting Hierarchical Structure Learning with Hyperbolic Distance Enhances Open World Object Detection](https://mlsanity.com/paper/2306.14291/) (2023)
- [Hyperbolic Dataset Distillation](https://mlsanity.com/paper/2505.24623/) (2025)
- [Hyperbolic Learning with Synthetic Captions for Open-World Detection](https://mlsanity.com/paper/2404.05016/) (2024)
- [Vector-Quantized Vision Foundation Models for Object-Centric Learning](https://mlsanity.com/paper/2502.20263/) (2025)
- [Understanding Hyperbolic Metric Learning through Hard Negative Sampling](https://mlsanity.com/paper/2404.15523/) (2024)
- [Hyperbolic Learning with Multimodal Large Language Models](https://mlsanity.com/paper/2408.05097/) (2024)
- [Learning Contrastive Self-Distillation for Ultra-Fine-Grained Visual Categorization Targeting Limited Samples](https://mlsanity.com/paper/2311.06056/) (2023)
- [Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval](https://mlsanity.com/paper/2411.17490/) (2024)
- [A Fashion Item Recommendation Model in Hyperbolic Space](https://mlsanity.com/paper/2409.02599/) (2024)
- [Multi-Scale Fusion for Object Representation](https://mlsanity.com/paper/2410.01539/) (2024)
- [HMSN: Hyperbolic Self-Supervised Learning by Clustering with Ideal Prototypes](https://mlsanity.com/paper/2305.10926/) (2023)
- [Hyperbolic vs Euclidean Embeddings in Few-Shot Learning: Two Sides of the Same Coin](https://mlsanity.com/paper/2309.10013/) (2023)
- [FairEHR-CLP: Towards Fairness-Aware Clinical Predictions with Contrastive Learning in Multimodal Electronic Health Records](https://mlsanity.com/paper/2402.00955/) (2024)
- [GeoWorld: Geometric World Models](https://mlsanity.com/paper/2602.23058/) (2026)
- [Continual Hyperbolic Learning of Instances and Classes](https://mlsanity.com/paper/2506.10710/) (2025)
- [HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Models](https://mlsanity.com/paper/2510.20322/) (2025)
- [A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets](https://mlsanity.com/paper/2403.06295/) (2024)
- [HypDAE: Hyperbolic Diffusion Autoencoders for Hierarchical Few-shot Image Generation](https://mlsanity.com/paper/2411.17784/) (2024)
- [Curvature Learning for Generalization of Hyperbolic Neural Networks](https://mlsanity.com/paper/2508.17232/) (2025)

## Abstract (arXiv)

Although self-/un-supervised methods have led to rapid progress in visual representation learning, these methods generally treat objects and scenes using the same lens. In this paper, we focus on learning representations for objects and scenes that preserve the structure among them. Motivated by the observation that visually similar objects are close in the representation space, we argue that the scenes and objects should instead follow a hierarchical structure based on their compositionality. To exploit such a structure, we propose a contrastive learning framework where a Euclidean loss is used to learn object representations and a hyperbolic loss is used to encourage representations of scenes to lie close to representations of their constituent objects in a hyperbolic space. This novel hyperbolic objective encourages the scene-object hypernymy among the representations by optimizing the magnitude of their norms. We show that when pretraining on the COCO and OpenImages datasets, the hyperbolic loss improves downstream performance of several baselines across multiple datasets and tasks, including image classification, object detection, and semantic segmentation. We also show that the properties of the learned representations allow us to solve various vision tasks that involve the interaction between scenes and objects in a zero-shot fashion. Our code can be found at urlhttps://github.com/shlokk/HCL/tree/main/HCL.