# Multi-Order Networks for Action Unit Detection

arXiv: https://arxiv.org/abs/2202.00446 · HTML: https://mlsanity.com/paper/2202.00446/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Order Matters: Sequence to sequence for sets](https://mlsanity.com/paper/1511.06391/) (2015)
- [UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory](https://mlsanity.com/paper/1609.02132/) (2016)
- [AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning](https://mlsanity.com/paper/1911.12423/) (2019)
- [Recurrent Neural Network for Text Classification with Multi-Task Learning](https://mlsanity.com/paper/1605.05101/) (2016)
- [NDDR-CNN: Layerwise Feature Fusing in Multi-Task CNNs by Neural Discriminative Dimensionality Reduction](https://mlsanity.com/paper/1801.08297/) (2018)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classification](https://mlsanity.com/paper/1611.05377/) (2016)
- [JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention](https://mlsanity.com/paper/2003.08834/) (2020)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/1904.09939/) (2019)
- [Facial Action Unit Detection Using Attention and Relation Learning](https://mlsanity.com/paper/1808.03457/) (2018)
- [Deep Structure Inference Network for Facial Action Unit Recognition](https://mlsanity.com/paper/1803.05873/) (2018)
- [Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering](https://mlsanity.com/paper/1711.00108/) (2017)
- [EAC-Net: A Region-based Deep Enhancing and Cropping Approach for Facial Action Unit Detection](https://mlsanity.com/paper/1702.02925/) (2017)
- [Integrated perception with recurrent multi-task neural networks](https://mlsanity.com/paper/1606.01735/) (2016)
- [Meta Auxiliary Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2105.06620/) (2021)

## Built on by (most cited first, 6 of 6)

- [Multi-label Transformer for Action Unit Detection](https://mlsanity.com/paper/2203.12531/) (2022)
- [FAN-Trans: Online Knowledge Distillation for Facial Action Unit Detection](https://mlsanity.com/paper/2211.06143/) (2022)
- [Fighting over-fitting with quantization for learning deep neural networks on noisy labels](https://mlsanity.com/paper/2303.11803/) (2023)
- [Attention Based Relation Network for Facial Action Units Recognition](https://mlsanity.com/paper/2210.13988/) (2022)
- [Fighting noise and imbalance in Action Unit detection problems](https://mlsanity.com/paper/2303.02994/) (2023)
- [Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction](https://mlsanity.com/paper/2310.05207/) (2023)

## Abstract (arXiv)

Action Units (AU) are muscular activations used to describe facial expressions. Therefore accurate AU recognition unlocks unbiaised face representation which can improve face-based affective computing applications. From a learning standpoint AU detection is a multi-task problem with strong inter-task dependencies. To solve such problem, most approaches either rely on weight sharing, or add explicit dependency modelling by decomposing the joint task distribution using Bayes chain rule. If the latter strategy yields comprehensive inter-task relationships modelling, it requires imposing an arbitrary order into an unordered task set. Crucially, this ordering choice has been identified as a source of performance variations. In this paper, we present Multi-Order Network (MONET), a multi-task method with joint task order optimization. MONET uses a differentiable order selection to jointly learn task-wise modules with their optimal chaining order. Furthermore, we introduce warmup and order dropout to enhance order selection by encouraging order exploration. Experimentally, we first demonstrate MONET capacity to retrieve the optimal order in a toy environment. Second, we validate MONET architecture by showing that MONET outperforms existing multi-task baselines on multiple attribute detection problems chosen for their wide range of dependency settings. More importantly, we demonstrate that MONET significantly extends state-of-the-art performance in AU detection.