# Learning Structured Representations by Embedding Class Hierarchy with Fast Optimal Transport

arXiv: https://arxiv.org/abs/2410.03052 · HTML: https://mlsanity.com/paper/2410.03052/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Fast Computation of Wasserstein Barycenters](https://mlsanity.com/paper/1310.4375/) (2013)
- [BREEDS: Benchmarks for Subpopulation Shift](https://mlsanity.com/paper/2008.04859/) (2020)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Deep Metric Learning with Hierarchical Triplet Loss](https://mlsanity.com/paper/1810.06951/) (2018)
- [Large-scale Multi-label Learning with Missing Labels](https://mlsanity.com/paper/1307.5101/) (2013)
- [Tree-Sliced Variants of Wasserstein Distances](https://mlsanity.com/paper/1902.00342/) (2019)
- [Hierarchy-based Image Embeddings for Semantic Image Retrieval](https://mlsanity.com/paper/1809.09924/) (2018)
- [Active Learning with Partial Feedback](https://mlsanity.com/paper/1802.07427/) (2018)
- [Learning Hierarchy Aware Features for Reducing Mistake Severity](https://mlsanity.com/paper/2207.12646/) (2022)
- [HIER: Metric Learning Beyond Class Labels via Hierarchical Regularization](https://mlsanity.com/paper/2212.14258/) (2022)
- [Scalable Nearest Neighbor Search for Optimal Transport](https://mlsanity.com/paper/1910.04126/) (2019)
- [Hierarchical Proxy-based Loss for Deep Metric Learning](https://mlsanity.com/paper/2103.13538/) (2021)
- [Embedding Label Structures for Fine-Grained Feature Representation](https://mlsanity.com/paper/1512.02895/) (2015)
- [Y-Tuning: An Efficient Tuning Paradigm for Large-Scale Pre-Trained Models via Label Representation Learning](https://mlsanity.com/paper/2202.09817/) (2022)
- [Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks](https://mlsanity.com/paper/1710.10393/) (2017)
- [Supervised Tree-Wasserstein Distance](https://mlsanity.com/paper/2101.11520/) (2021)
- [Approximating 1-Wasserstein Distance with Trees](https://mlsanity.com/paper/2206.12116/) (2022)
- [Efficient Loss-Based Decoding on Graphs For Extreme Classification](https://mlsanity.com/paper/1803.03319/) (2018)
- [Accelerating Extreme Classification via Adaptive Feature Agglomeration](https://mlsanity.com/paper/1905.11769/) (2019)
- [SEAL: Simultaneous Label Hierarchy Exploration And Learning](https://mlsanity.com/paper/2304.13374/) (2023)
- [Embedding Semantic Hierarchy in Discrete Optimal Transport for Risk Minimization](https://mlsanity.com/paper/2105.00101/) (2021)
- [Rank-based loss for learning hierarchical representations](https://mlsanity.com/paper/2110.05941/) (2021)

## Built on by (most cited first, 3 of 3)

- [The LLM Bottleneck: Why Open-Source Vision LLMs Struggle with Hierarchical Visual Recognition](https://mlsanity.com/paper/2505.24840/) (2025)
- [Taxonomy-Aware Representation Alignment for Hierarchical Visual Recognition with Large Multimodal Models](https://mlsanity.com/paper/2603.00431/) (2026)
- [Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models](https://mlsanity.com/paper/2607.02909/) (2026)

## Abstract (arXiv)

To embed structured knowledge within labels into feature representations, prior work [Zeng et al., 2022] proposed to use the Cophenetic Correlation Coefficient (CPCC) as a regularizer during supervised learning. This regularizer calculates pairwise Euclidean distances of class means and aligns them with the corresponding shortest path distances derived from the label hierarchy tree. However, class means may not be good representatives of the class conditional distributions, especially when they are multi-mode in nature. To address this limitation, under the CPCC framework, we propose to use the Earth Mover's Distance (EMD) to measure the pairwise distances among classes in the feature space. We show that our exact EMD method generalizes previous work, and recovers the existing algorithm when class-conditional distributions are Gaussian. To further improve the computational efficiency of our method, we introduce the Optimal Transport-CPCC family by exploring four EMD approximation variants. Our most efficient OT-CPCC variant, the proposed Fast FlowTree algorithm, runs in linear time in the size of the dataset, while maintaining competitive performance across datasets and tasks. The code is available at https://github.com/uiuctml/OTCPCC.