# Architecture Disentanglement for Deep Neural Networks

arXiv: https://arxiv.org/abs/2003.13268 · HTML: https://mlsanity.com/paper/2003.13268/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2020.

## Built on by (most cited first, 10 of 10)

- [Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement](https://mlsanity.com/paper/2203.08008/) (2022)
- [Learning Task-preferred Inference Routes for Gradient De-conflict in Multi-output DNNs](https://mlsanity.com/paper/2305.19844/) (2023)
- [On the Relationship Between Interpretability and Explainability in Machine Learning](https://mlsanity.com/paper/2311.11491/) (2023)
- [Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation](https://mlsanity.com/paper/2308.12053/) (2023)
- [Deep Learning Based Decision Support for Medicine -- A Case Study on Skin Cancer Diagnosis](https://mlsanity.com/paper/2103.05112/) (2021)
- [Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks](https://mlsanity.com/paper/2203.13453/) (2022)
- [Quantifying Human Bias and Knowledge to guide ML models during Training](https://mlsanity.com/paper/2211.10796/) (2022)
- [Mutual Understanding between People and Systems via Neurosymbolic AI and Knowledge Graphs](https://mlsanity.com/paper/2504.11200/) (2025)
- [MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction](https://mlsanity.com/paper/2409.09446/) (2024)
- [FAMES: Fast Approximate Multiplier Substitution for Mixed-Precision Quantized DNNs--Down to 2 Bits!](https://mlsanity.com/paper/2411.18055/) (2024)

## Abstract (arXiv)

Understanding the inner workings of deep neural networks (DNNs) is essential to provide trustworthy artificial intelligence techniques for practical applications. Existing studies typically involve linking semantic concepts to units or layers of DNNs, but fail to explain the inference process. In this paper, we introduce neural architecture disentanglement (NAD) to fill the gap. Specifically, NAD learns to disentangle a pre-trained DNN into sub-architectures according to independent tasks, forming information flows that describe the inference processes. We investigate whether, where, and how the disentanglement occurs through experiments conducted with handcrafted and automatically-searched network architectures, on both object-based and scene-based datasets. Based on the experimental results, we present three new findings that provide fresh insights into the inner logic of DNNs. First, DNNs can be divided into sub-architectures for independent tasks. Second, deeper layers do not always correspond to higher semantics. Third, the connection type in a DNN affects how the information flows across layers, leading to different disentanglement behaviors. With NAD, we further explain why DNNs sometimes give wrong predictions. Experimental results show that misclassified images have a high probability of being assigned to task sub-architectures similar to the correct ones. Code will be available at: https://github.com/hujiecpp/NAD.