# Deep Dependency Networks for Multi-Label Classification

arXiv: https://arxiv.org/abs/2302.00633 · HTML: https://mlsanity.com/paper/2302.00633/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 101 references that are in the map.
- First posted on arXiv 1 Feb 2023.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Conditional Random Fields as Recurrent Neural Networks](https://mlsanity.com/paper/1502.03240/) (2015)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [Query2Label: A Simple Transformer Way to Multi-Label Classification](https://mlsanity.com/paper/2107.10834/) (2021)
- [General Multi-label Image Classification with Transformers](https://mlsanity.com/paper/2011.14027/) (2020)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [Deep Convolutional Neural Fields for Depth Estimation from a Single Image](https://mlsanity.com/paper/1411.6387/) (2014)
- [Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification](https://mlsanity.com/paper/1702.05891/) (2017)
- [Multi-label Image Recognition by Recurrently Discovering Attentional Regions](https://mlsanity.com/paper/1711.02816/) (2017)
- [Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2012.02994/) (2020)
- [Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification](https://mlsanity.com/paper/1912.07872/) (2019)
- [Efficient piecewise training of deep structured models for semantic segmentation](https://mlsanity.com/paper/1504.01013/) (2015)
- [CNN: Single-label to Multi-label](https://mlsanity.com/paper/1406.5726/) (2014)
- [Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition](https://mlsanity.com/paper/2007.01755/) (2020)
- [Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition](https://mlsanity.com/paper/1712.07465/) (2017)
- [Thin-Slicing Network: A Deep Structured Model for Pose Estimation in Videos](https://mlsanity.com/paper/1703.10898/) (2017)
- [Exploit Bounding Box Annotations for Multi-label Object Recognition](https://mlsanity.com/paper/1504.05843/) (2015)

## Built on by (most cited first, 1 of 1)

- [Multidimensional classification of posts for online course discussion forum curation](https://mlsanity.com/paper/2508.10008/) (2025)

## Abstract (arXiv)

We propose a simple approach which combines the strengths of probabilistic graphical models and deep learning architectures for solving the multi-label classification task, focusing specifically on image and video data. First, we show that the performance of previous approaches that combine Markov Random Fields with neural networks can be modestly improved by leveraging more powerful methods such as iterative join graph propagation, integer linear programming, and ℓ1 regularization-based structure learning. Then we propose a new modeling framework called deep dependency networks, which augments a dependency network, a model that is easy to train and learns more accurate dependencies but is limited to Gibbs sampling for inference, to the output layer of a neural network. We show that despite its simplicity, jointly learning this new architecture yields significant improvements in performance over the baseline neural network. In particular, our experimental evaluation on three video activity classification datasets: Charades, Textually Annotated Cooking Scenes (TACoS), and Wetlab, and three multi-label image classification datasets: MS-COCO, PASCAL VOC, and NUS-WIDE show that deep dependency networks are almost always superior to pure neural architectures that do not use dependency networks.