# Digging Deeper: Learning Multi-Level Concept Hierarchies

arXiv: https://arxiv.org/abs/2603.10084 · HTML: https://mlsanity.com/paper/2603.10084/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [BatchTopK Sparse Autoencoders](https://mlsanity.com/paper/2412.06410/) (2024)
- [Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networks](https://mlsanity.com/paper/1801.03454/) (2018)
- [Stochastic Concept Bottleneck Models](https://mlsanity.com/paper/2406.19272/) (2024)
- [Causal Concept Graph Models: Beyond Causal Opacity in Deep Learning](https://mlsanity.com/paper/2405.16507/) (2024)
- [Incorporating Hierarchical Semantics in Sparse Autoencoder Architectures](https://mlsanity.com/paper/2506.01197/) (2025)
- [Understanding Inter-Concept Relationships in Concept-Based Models](https://mlsanity.com/paper/2405.18217/) (2024)
- [Avoiding Leakage Poisoning: Concept Interventions Under Distribution Shifts](https://mlsanity.com/paper/2504.17921/) (2025)
- [Addressing Concept Mislabeling in Concept Bottleneck Models Through Preference Optimization](https://mlsanity.com/paper/2504.18026/) (2025)
- [Hierarchical Concept-based Interpretable Models](https://mlsanity.com/paper/2602.23947/) (2026)

## Abstract (arXiv)

Although concept-based models promise interpretability by explaining predictions with human-understandable concepts, they typically rely on exhaustive annotations and treat concepts as flat and independent. To circumvent this, recent work has introduced Hierarchical Concept Embedding Models (HiCEMs) to explicitly model concept relationships, and Concept Splitting to discover sub-concepts using only coarse annotations. However, both HiCEMs and Concept Splitting are restricted to shallow hierarchies. We overcome this limitation with Multi-Level Concept Splitting (MLCS), which discovers multi-level concept hierarchies from only top-level supervision, and Deep-HiCEMs, an architecture that represents these discovered hierarchies and enables interventions at multiple levels of abstraction. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.