# On Smart Gaze based Annotation of Histopathology Images for Training of Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2202.02764 · HTML: https://mlsanity.com/paper/2202.02764/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 38 references that are in the map.
- First posted on arXiv 6 Feb 2022.

## Builds on (most cited first, 6 of 6)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks](https://mlsanity.com/paper/2001.09521/) (2020)
- [Gaze2Segment: A Pilot Study for Integrating Eye-Tracking Technology into Medical Image Segmentation](https://mlsanity.com/paper/1608.03235/) (2016)
- [Basic Ensembles of Vanilla-Style Deep Learning Models Improve Liver Segmentation From CT Images](https://mlsanity.com/paper/2001.09647/) (2020)
- [Expected exponential loss for gaze-based video and volume ground truth annotation](https://mlsanity.com/paper/1707.04905/) (2017)

## Built on by (most cited first, 2 of 2)

- [When Eye-Tracking Meets Machine Learning: A Systematic Review on Applications in Medical Image Analysis](https://mlsanity.com/paper/2403.07834/) (2024)
- [Supporting Mitosis Detection AI Training with Inter-Observer Eye-Gaze Consistencies](https://mlsanity.com/paper/2404.01656/) (2024)

## Abstract (arXiv)

Unavailability of large training datasets is a bottleneck that needs to be overcome to realize the true potential of deep learning in histopathology applications. Although slide digitization via whole slide imaging scanners has increased the speed of data acquisition, labeling of virtual slides requires a substantial time investment from pathologists. Eye gaze annotations have the potential to speed up the slide labeling process. This work explores the viability and timing comparisons of eye gaze labeling compared to conventional manual labeling for training object detectors. Challenges associated with gaze based labeling and methods to refine the coarse data annotations for subsequent object detection are also discussed. Results demonstrate that gaze tracking based labeling can save valuable pathologist time and delivers good performance when employed for training a deep object detector. Using the task of localization of Keratin Pearls in cases of oral squamous cell carcinoma as a test case, we compare the performance gap between deep object detectors trained using hand-labelled and gaze-labelled data. On average, compared to `Bounding-box' based hand-labeling, gaze-labeling required 57.6% less time per label and compared to `Freehand' labeling, gaze-labeling required on average 85% less time per label.