# Incremental Boosting Convolutional Neural Network for Facial Action Unit Recognition

arXiv: https://arxiv.org/abs/1707.05395 · HTML: https://mlsanity.com/paper/1707.05395/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jul 2017.

## Built on by (most cited first, 16 of 16)

- [Ensemble deep learning: A review](https://mlsanity.com/paper/2104.02395/) (2021)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges](https://mlsanity.com/paper/2303.01498/) (2023)
- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition](https://mlsanity.com/paper/2407.03835/) (2024)
- [A step towards understanding why classification helps regression](https://mlsanity.com/paper/2308.10603/) (2023)
- [Lifelong Ensemble Learning based on Multiple Representations for Few-Shot Object Recognition](https://mlsanity.com/paper/2205.01982/) (2022)
- [Improved Crop and Weed Detection with Diverse Data Ensemble Learning](https://mlsanity.com/paper/2310.01055/) (2023)
- [Synergy Between Semantic Segmentation and Image Denoising via Alternate Boosting](https://mlsanity.com/paper/2102.12095/) (2021)
- [Dual Convexified Convolutional Neural Networks](https://mlsanity.com/paper/2205.14056/) (2022)
- [Tricks and Plugins to GBM on Images and Sequences](https://mlsanity.com/paper/2203.00761/) (2022)
- [HCE: Improving Performance and Efficiency with Heterogeneously Compressed Neural Network Ensemble](https://mlsanity.com/paper/2301.07794/) (2023)
- [Deep Boosting Multi-Modal Ensemble Face Recognition with Sample-Level Weighting](https://mlsanity.com/paper/2308.09234/) (2023)
- [Tricks and Plug-ins for Gradient Boosting in Image Classification](https://mlsanity.com/paper/2507.22842/) (2025)

## Abstract (arXiv)

Recognizing facial action units (AUs) from spontaneous facial expressions is still a challenging problem. Most recently, CNNs have shown promise on facial AU recognition. However, the learned CNNs are often overfitted and do not generalize well to unseen subjects due to limited AU-coded training images. We proposed a novel Incremental Boosting CNN (IB-CNN) to integrate boosting into the CNN via an incremental boosting layer that selects discriminative neurons from the lower layer and is incrementally updated on successive mini-batches. In addition, a novel loss function that accounts for errors from both the incremental boosted classifier and individual weak classifiers was proposed to fine-tune the IB-CNN. Experimental results on four benchmark AU databases have demonstrated that the IB-CNN yields significant improvement over the traditional CNN and the boosting CNN without incremental learning, as well as outperforming the state-of-the-art CNN-based methods in AU recognition. The improvement is more impressive for the AUs that have the lowest frequencies in the databases.