# Joint Hierarchical Category Structure Learning and Large-Scale Image Classification

arXiv: https://arxiv.org/abs/1709.05072 · HTML: https://mlsanity.com/paper/1709.05072/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2017.

## Built on by (most cited first, 6 of 6)

- [Hierarchical Few-Shot Object Detection: Problem, Benchmark and Method](https://mlsanity.com/paper/2210.03940/) (2022)
- [SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery](https://mlsanity.com/paper/2510.18740/) (2025)
- [MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classification](https://mlsanity.com/paper/2107.00808/) (2021)
- [Encoding Hierarchical Information in Neural Networks helps in Subpopulation Shift](https://mlsanity.com/paper/2112.10844/) (2021)
- [Pitfalls of Assessing Extracted Hierarchies for Multi-Class Classification](https://mlsanity.com/paper/2101.11095/) (2021)
- [Categorical Knowledge Fused Recognition: Fusing Hierarchical Knowledge with Image Classification through Aligning and Deep Metric Learning](https://mlsanity.com/paper/2407.20600/) (2024)

## Abstract (arXiv)

We investigate the scalable image classification problem with a large number of categories. Hierarchical visual data structures are helpful for improving the efficiency and performance of large-scale multi-class classification. We propose a novel image classification method based on learning hierarchical inter-class structures. Specifically, we first design a fast algorithm to compute the similarity metric between categories, based on which a visual tree is constructed by hierarchical spectral clustering. Using the learned visual tree, a test sample label is efficiently predicted by searching for the best path over the entire tree. The proposed method is extensively evaluated on the ILSVRC2010 and Caltech 256 benchmark datasets. Experimental results show that our method obtains significantly better category hierarchies than other state-of-the-art visual tree-based methods and, therefore, much more accurate classification.