# MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classification

arXiv: https://arxiv.org/abs/2107.00808 · HTML: https://mlsanity.com/paper/2107.00808/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 2 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Video Playback Rate Perception for Self-supervisedSpatio-Temporal Representation Learning](https://mlsanity.com/paper/2006.11476/) (2020)
- [Video Cloze Procedure for Self-Supervised Spatio-Temporal Learning](https://mlsanity.com/paper/2001.00294/) (2020)
- [Network of Experts for Large-Scale Image Categorization](https://mlsanity.com/paper/1604.06119/) (2016)
- [Hierarchy-based Image Embeddings for Semantic Image Retrieval](https://mlsanity.com/paper/1809.09924/) (2018)
- [Heterogeneous Multi-task Learning for Human Pose Estimation with Deep Convolutional Neural Network](https://mlsanity.com/paper/1406.3474/) (2014)
- [Curved Text Detection in Natural Scene Images with Semi- and Weakly-Supervised Learning](https://mlsanity.com/paper/1908.09990/) (2019)
- [Self-Training for Domain Adaptive Scene Text Detection](https://mlsanity.com/paper/2005.11487/) (2020)
- [Exploring Relations in Untrimmed Videos for Self-Supervised Learning](https://mlsanity.com/paper/2008.02711/) (2020)
- [Gaussian Constrained Attention Network for Scene Text Recognition](https://mlsanity.com/paper/2010.09169/) (2020)
- [FC2RN: A Fully Convolutional Corner Refinement Network for Accurate Multi-Oriented Scene Text Detection](https://mlsanity.com/paper/2007.05113/) (2020)
- [Progressive Cluster Purification for Unsupervised Feature Learning](https://mlsanity.com/paper/2007.02577/) (2020)
- [Two-Level Residual Distillation based Triple Network for Incremental Object Detection](https://mlsanity.com/paper/2007.13428/) (2020)
- [Joint Hierarchical Category Structure Learning and Large-Scale Image Classification](https://mlsanity.com/paper/1709.05072/) (2017)
- [Visual Tree Convolutional Neural Network in Image Classification](https://mlsanity.com/paper/1906.01536/) (2019)
- [Exploring Instance Relations for Unsupervised Feature Embedding](https://mlsanity.com/paper/2105.03341/) (2021)
- [Visual Confusion Label Tree For Image Classification](https://mlsanity.com/paper/1906.02012/) (2019)

## Built on by (most cited first, 5 of 5)

- [PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition](https://mlsanity.com/paper/2109.04145/) (2021)
- [Dense Semantic Contrast for Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2109.07756/) (2021)
- [Revisiting Single-gated Mixtures of Experts](https://mlsanity.com/paper/2304.05497/) (2023)
- [Performance Improvement in Multi-class Classification via Automated Hierarchy Generation and Exploitation through Extended LCPN Schemes](https://mlsanity.com/paper/2310.20641/) (2023)
- [Hierarchy-Aware Fine-Tuning of Vision-Language Models](https://mlsanity.com/paper/2512.21529/) (2025)

## Abstract (arXiv)

Hierarchical classification is significant for complex tasks by providing multi-granular predictions and encouraging better mistakes. As the label structure decides its performance, many existing approaches attempt to construct an excellent label structure for promoting the classification results. In this paper, we consider that different label structures provide a variety of prior knowledge for category recognition, thus fusing them is helpful to achieve better hierarchical classification results. Furthermore, we propose a multi-task multi-structure fusion model to integrate different label structures. It contains two kinds of branches: one is the traditional classification branch to classify the common subclasses, the other is responsible for identifying the heterogeneous superclasses defined by different label structures. Besides the effect of multiple label structures, we also explore the architecture of the deep model for better hierachical classification and adjust the hierarchical evaluation metrics for multiple label structures. Experimental results on CIFAR100 and Car196 show that our method obtains significantly better results than using a flat classifier or a hierarchical classifier with any single label structure.