# Trunk-Branch Ensemble Convolutional Neural Networks for Video-based Face Recognition

arXiv: https://arxiv.org/abs/1607.05427 · HTML: https://mlsanity.com/paper/1607.05427/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jul 2016.

## Built on by (most cited first, 16 of 16)

- [Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On](https://mlsanity.com/paper/2404.01089/) (2024)
- [Towards Variable and Coordinated Holistic Co-Speech Motion Generation](https://mlsanity.com/paper/2404.00368/) (2024)
- [Part-based Face Recognition with Vision Transformers](https://mlsanity.com/paper/2212.00057/) (2022)
- [LAFS: Landmark-based Facial Self-supervised Learning for Face Recognition](https://mlsanity.com/paper/2403.08161/) (2024)
- [Recent Advances in Deep Learning Techniques for Face Recognition](https://mlsanity.com/paper/2103.10492/) (2021)
- [Reasoning Graph Networks for Kinship Verification: from Star-shaped to Hierarchical](https://mlsanity.com/paper/2109.02219/) (2021)
- [Modularity in Deep Learning: A Survey](https://mlsanity.com/paper/2310.01154/) (2023)
- [A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture](https://mlsanity.com/paper/2101.03308/) (2021)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [On the Stability and Generalization of Triplet Learning](https://mlsanity.com/paper/2302.09815/) (2023)
- [Kinship Representation Learning with Face Componential Relation](https://mlsanity.com/paper/2304.04546/) (2023)
- [FabricNet: A Fiber Recognition Architecture Using Ensemble ConvNets](https://mlsanity.com/paper/2101.05564/) (2021)
- [ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer](https://mlsanity.com/paper/2510.11073/) (2025)
- [Dynamic Distribution of Edge Intelligence at the Node Level for Internet of Things](https://mlsanity.com/paper/2107.05828/) (2021)
- [RRR-Net: Reusing, Reducing, and Recycling a Deep Backbone Network](https://mlsanity.com/paper/2310.01157/) (2023)
- [Video-Based Palm-Vein Authentication under Challenging Conditions](https://mlsanity.com/paper/2609.02776/) (2026)

## Abstract (arXiv)

Human faces in surveillance videos often suffer from severe image blur, dramatic pose variations, and occlusion. In this paper, we propose a comprehensive framework based on Convolutional Neural Networks (CNN) to overcome challenges in video-based face recognition (VFR). First, to learn blur-robust face representations, we artificially blur training data composed of clear still images to account for a shortfall in real-world video training data. Using training data composed of both still images and artificially blurred data, CNN is encouraged to learn blur-insensitive features automatically. Second, to enhance robustness of CNN features to pose variations and occlusion, we propose a Trunk-Branch Ensemble CNN model (TBE-CNN), which extracts complementary information from holistic face images and patches cropped around facial components. TBE-CNN is an end-to-end model that extracts features efficiently by sharing the low- and middle-level convolutional layers between the trunk and branch networks. Third, to further promote the discriminative power of the representations learnt by TBE-CNN, we propose an improved triplet loss function. Systematic experiments justify the effectiveness of the proposed techniques. Most impressively, TBE-CNN achieves state-of-the-art performance on three popular video face databases: PaSC, COX Face, and YouTube Faces. With the proposed techniques, we also obtain the first place in the BTAS 2016 Video Person Recognition Evaluation.