# Hierarchical Image Classification using Entailment Cone Embeddings

arXiv: https://arxiv.org/abs/2004.03459 · HTML: https://mlsanity.com/paper/2004.03459/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 30 of 46)

- [Hyperbolic Image Segmentation](https://mlsanity.com/paper/2203.05898/) (2022)
- [Hyperbolic Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/2305.06611/) (2023)
- [Learning the Predictability of the Future](https://mlsanity.com/paper/2101.01600/) (2021)
- [Hypformer: Exploring Efficient Transformer Fully in Hyperbolic Space](https://mlsanity.com/paper/2407.01290/) (2024)
- [Poincar'e ResNet](https://mlsanity.com/paper/2303.14027/) (2023)
- [HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding](https://mlsanity.com/paper/2311.14064/) (2023)
- [From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding](https://mlsanity.com/paper/2304.00553/) (2023)
- [Emergent Visual-Semantic Hierarchies in Image-Text Representations](https://mlsanity.com/paper/2407.08521/) (2024)
- [HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts](https://mlsanity.com/paper/2404.17507/) (2024)
- [Learning Structured Representations with Hyperbolic Embeddings](https://mlsanity.com/paper/2412.01023/) (2024)
- [Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition](https://mlsanity.com/paper/2107.07029/) (2021)
- [Taxonomy-Aware Evaluation of Vision-Language Models](https://mlsanity.com/paper/2504.05457/) (2025)
- [Hierarchical Knowledge Guided Learning for Real-world Retinal Diseases Recognition](https://mlsanity.com/paper/2111.08913/) (2021)
- [Optimization of Rank Losses for Image Retrieval](https://mlsanity.com/paper/2309.08250/) (2023)
- [Geometric Relational Embeddings: A Survey](https://mlsanity.com/paper/2304.11949/) (2023)
- [Hierarchical Average Precision Training for Pertinent Image Retrieval](https://mlsanity.com/paper/2207.04873/) (2022)
- [Learning Visual Hierarchies in Hyperbolic Space for Image Retrieval](https://mlsanity.com/paper/2411.17490/) (2024)
- [A Survey of Neural Trees](https://mlsanity.com/paper/2209.03415/) (2022)
- [Low-distortion and GPU-compatible Tree Embeddings in Hyperbolic Space](https://mlsanity.com/paper/2502.17130/) (2025)
- [Hyperbolic Large Language Models](https://mlsanity.com/paper/2509.05757/) (2025)
- [Continual Hyperbolic Learning of Instances and Classes](https://mlsanity.com/paper/2506.10710/) (2025)
- [Digital Taxonomist: Identifying Plant Species in Community Scientists' Photographs](https://mlsanity.com/paper/2106.03774/) (2021)
- [What is Your Metric Telling You? Evaluating Classifier Calibration under Context-Specific Definitions of Reliability](https://mlsanity.com/paper/2205.11454/) (2022)
- [A Capsule Network for Hierarchical Multi-Label Image Classification](https://mlsanity.com/paper/2209.05723/) (2022)
- [One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spaces](https://mlsanity.com/paper/2512.00852/) (2025)
- [United We Learn Better: Harvesting Learning Improvements From Class Hierarchies Across Tasks](https://mlsanity.com/paper/2107.13627/) (2021)
- [Label Hierarchy Transition: Delving into Class Hierarchies to Enhance Deep Classifiers](https://mlsanity.com/paper/2112.02353/) (2021)
- [Encoding Hierarchical Information in Neural Networks helps in Subpopulation Shift](https://mlsanity.com/paper/2112.10844/) (2021)
- [All Mistakes Are Not Equal: Comprehensive Hierarchy Aware Multi-label Predictions (CHAMP)](https://mlsanity.com/paper/2206.08653/) (2022)
- [Adversarial Attacks on Hyperbolic Networks](https://mlsanity.com/paper/2412.01495/) (2024)

## Abstract (arXiv)

Image classification has been studied extensively, but there has been limited work in using unconventional, external guidance other than traditional image-label pairs for training. We present a set of methods for leveraging information about the semantic hierarchy embedded in class labels. We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier and empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance. Taking a step further in this direction, we model more explicitly the label-label and label-image interactions using order-preserving embeddings governed by both Euclidean and hyperbolic geometries, prevalent in natural language, and tailor them to hierarchical image classification and representation learning. We empirically validate all the models on the hierarchical ETHEC dataset.