# BIFRNet: A Brain-Inspired Feature Restoration DNN for Partially Occluded Image Recognition

arXiv: https://arxiv.org/abs/2303.01309 · HTML: https://mlsanity.com/paper/2303.01309/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers](https://mlsanity.com/paper/2103.12340/) (2021)
- [Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network](https://mlsanity.com/paper/1908.06290/) (2019)
- [Mask-Guided Attention Network for Occluded Pedestrian Detection](https://mlsanity.com/paper/1910.06160/) (2019)
- [Comparing deep neural networks against humans: object recognition when the signal gets weaker](https://mlsanity.com/paper/1706.06969/) (2017)
- [Combining Compositional Models and Deep Networks For Robust Object Classification under Occlusion](https://mlsanity.com/paper/1905.11826/) (2019)
- [End2End Occluded Face Recognition by Masking Corrupted Features](https://mlsanity.com/paper/2108.09468/) (2021)
- [Capturing the objects of vision with neural networks](https://mlsanity.com/paper/2109.03351/) (2021)
- [Face De-occlusion using 3D Morphable Model and Generative Adversarial Network](https://mlsanity.com/paper/1904.06109/) (2019)
- [Detecting Semantic Parts on Partially Occluded Objects](https://mlsanity.com/paper/1707.07819/) (2017)
- [Towards Deep Compositional Networks](https://mlsanity.com/paper/1609.03795/) (2016)
- [SD-GAN: Structural and Denoising GAN reveals facial parts under occlusion](https://mlsanity.com/paper/2002.08448/) (2020)

## Built on by (most cited first, 1 of 1)

- [Robustness of SAM: Segment Anything Under Corruptions and Beyond](https://mlsanity.com/paper/2306.07713/) (2023)

## Abstract (arXiv)

The partially occluded image recognition (POIR) problem has been a challenge for artificial intelligence for a long time. A common strategy to handle the POIR problem is using the non-occluded features for classification. Unfortunately, this strategy will lose effectiveness when the image is severely occluded, since the visible parts can only provide limited information. Several studies in neuroscience reveal that feature restoration which fills in the occluded information and is called amodal completion is essential for human brains to recognize partially occluded images. However, feature restoration is commonly ignored by CNNs, which may be the reason why CNNs are ineffective for the POIR problem. Inspired by this, we propose a novel brain-inspired feature restoration network (BIFRNet) to solve the POIR problem. It mimics a ventral visual pathway to extract image features and a dorsal visual pathway to distinguish occluded and visible image regions. In addition, it also uses a knowledge module to store object prior knowledge and uses a completion module to restore occluded features based on visible features and prior knowledge. Thorough experiments on synthetic and real-world occluded image datasets show that BIFRNet outperforms the existing methods in solving the POIR problem. Especially for severely occluded images, BIRFRNet surpasses other methods by a large margin and is close to the human brain performance. Furthermore, the brain-inspired design makes BIFRNet more interpretable.