# VeriMedi: Pill Identification using Proxy-based Deep Metric Learning and Exact Solution

arXiv: https://arxiv.org/abs/2104.11231 · HTML: https://mlsanity.com/paper/2104.11231/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [The VIA Annotation Software for Images, Audio and Video](https://mlsanity.com/paper/1904.10699/) (2019)
- [Explainable Artificial Intelligence: a Systematic Review](https://mlsanity.com/paper/2006.00093/) (2020)
- [PyTorch Metric Learning](https://mlsanity.com/paper/2008.09164/) (2020)
- [ePillID Dataset: A Low-Shot Fine-Grained Benchmark for Pill Identification](https://mlsanity.com/paper/2005.14288/) (2020)

## Abstract (arXiv)

We present the system that we have developed for the identification and verification of pills using images that are taken by the VeriMedi device. The VeriMedi device is an Internet of Things device that takes pictures of a filled pill vial from the bottom of the vial and uses the solution that is presented in this research to identify the pills in the vials. The solution has two serially connected deep learning solutions which do segmentation and identification. The segmentation solution creates the masks for each pill in the vial image by using the Mask R-CNN model, then segments and crops the pills and blurs the background. After that, the segmented pill images are sent to the identification solution where a Deep Metric Learning model that is trained with Proxy Anchor Loss (PAL) function generates embedding vectors for each pill image. The generated embedding vectors are fed into a one-layer fully connected network that is trained with the exact solution to predict each single pill image. Then, the aggregation/verification function aggregates the multiple predictions coming from multiple single pill images and verifies the correctness of the final prediction with respect to predefined rules. Besides, we enhanced the PAL with a better proxy initialization that increased the performance of the models and let the model learn the new classes of images continually without retraining the model with the whole dataset. When the model that is trained with initial classes is retrained only with new classes, the accuracy of the model increases for both old and new classes. The identification solution that we have presented in this research can also be reused for other problem domains which require continual learning and/or Fine-Grained Visual Categorization.