# Towards an Efficient Semantic Segmentation Method of ID Cards for Verification Systems

arXiv: https://arxiv.org/abs/2111.12764 · HTML: https://mlsanity.com/paper/2111.12764/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [MIDV-500: A Dataset for Identity Documents Analysis and Recognition on Mobile Devices in Video Stream](https://mlsanity.com/paper/1807.05786/) (2018)
- [Illuminating Pedestrians via Simultaneous Detection & Segmentation](https://mlsanity.com/paper/1706.08564/) (2017)
- [A Fast Fully Octave Convolutional Neural Network for Document Image Segmentation](https://mlsanity.com/paper/2004.01317/) (2020)
- [An Automatic Reader of Identity Documents](https://mlsanity.com/paper/2006.14853/) (2020)
- [Approach for Document Detection by Contours and Contrasts](https://mlsanity.com/paper/2008.02615/) (2020)
- [Advanced Hough-based method for on-device document localization](https://mlsanity.com/paper/2106.09987/) (2021)

## Built on by (most cited first, 3 of 3)

- [Synthetic ID Card Image Generation for Improving Presentation Attack Detection](https://mlsanity.com/paper/2211.00098/) (2022)
- [Improving Presentation Attack Detection for ID Cards on Remote Verification Systems](https://mlsanity.com/paper/2301.09542/) (2023)
- [Open-Set: ID Card Presentation Attack Detection using Neural Transfer Style](https://mlsanity.com/paper/2312.13993/) (2023)

## Abstract (arXiv)

Removing the background in ID Card images is a real challenge for remote verification systems because many of the re-digitalised images present cluttered backgrounds, poor illumination conditions, distortion and occlusions. The background in ID Card images confuses the classifiers and the text extraction. Due to the lack of available images for research, this field represents an open problem in computer vision today. This work proposes a method for removing the background using semantic segmentation of ID Cards. In the end, images captured in the wild from the real operation, using a manually labelled dataset consisting of 45,007 images, with five types of ID Cards from three countries (Chile, Argentina and Mexico), including typical presentation attack scenarios, were used. This method can help to improve the following stages in a regular identity verification or document tampering detection system. Two Deep Learning approaches were explored, based on MobileUNet and DenseNet10. The best results were obtained using MobileUNet, with 6.5 million parameters. A Chilean ID Card's mean Intersection Over Union (IoU) was 0.9926 on a private test dataset of 4,988 images. The best results for the fused multi-country dataset of ID Card images from Chile, Argentina and Mexico reached an IoU of 0.9911. The proposed methods are lightweight enough to be used in real-time operation on mobile devices.