# ViTree: Single-path Neural Tree for Step-wise Interpretable Fine-grained Visual Categorization

arXiv: https://arxiv.org/abs/2401.17050 · HTML: https://mlsanity.com/paper/2401.17050/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [TransFG: A Transformer Architecture for Fine-grained Recognition](https://mlsanity.com/paper/2103.07976/) (2021)
- [Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes](https://mlsanity.com/paper/2111.15000/) (2021)
- [Interpretable Image Classification with Differentiable Prototypes Assignment](https://mlsanity.com/paper/2112.02902/) (2021)
- [Adaptive Neural Trees](https://mlsanity.com/paper/1807.06699/) (2018)
- [ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition](https://mlsanity.com/paper/2208.10431/) (2022)
- [Feature Fusion Vision Transformer for Fine-Grained Visual Categorization](https://mlsanity.com/paper/2107.02341/) (2021)
- [Learning Support and Trivial Prototypes for Interpretable Image Classification](https://mlsanity.com/paper/2301.04011/) (2023)
- [Fine-grained Visual Classification with High-temperature Refinement and Background Suppression](https://mlsanity.com/paper/2303.06442/) (2023)
- [A free lunch from ViT:Adaptive Attention Multi-scale Fusion Transformer for Fine-grained Visual Recognition](https://mlsanity.com/paper/2110.01240/) (2021)
- [Fine-Grained Visual Classification using Self Assessment Classifier](https://mlsanity.com/paper/2205.10529/) (2022)
- [Take 5: Interpretable Image Classification with a Handful of Features](https://mlsanity.com/paper/2303.13166/) (2023)

## Built on by (most cited first, 1 of 1)

- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)

## Abstract (arXiv)

As computer vision continues to advance and finds widespread applications across various domains, the need for interpretability in deep learning models becomes paramount. Existing methods often resort to post-hoc techniques or prototypes to explain the decision-making process, which can be indirect and lack intrinsic illustration. In this research, we introduce ViTree, a novel approach for fine-grained visual categorization that combines the popular vision transformer as a feature extraction backbone with neural decision trees. By traversing the tree paths, ViTree effectively selects patches from transformer-processed features to highlight informative local regions, thereby refining representations in a step-wise manner. Unlike previous tree-based models that rely on soft distributions or ensembles of paths, ViTree selects a single tree path, offering a clearer and simpler decision-making process. This patch and path selectivity enhances model interpretability of ViTree, enabling better insights into the model's inner workings. Remarkably, extensive experimentation validates that this streamlined approach surpasses various strong competitors and achieves state-of-the-art performance while maintaining exceptional interpretability which is proved by multi-perspective methods. Code can be found at https://github.com/SJTU-DeepVisionLab/ViTree.