# Embracing Annotation Efficient Learning (AEL) for Digital Pathology and Natural Images

arXiv: https://arxiv.org/abs/2212.00470 · HTML: https://mlsanity.com/paper/2212.00470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 125 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [Rethinking Pre-training and Self-training](https://mlsanity.com/paper/2006.06882/) (2020)
- [Fixing the train-test resolution discrepancy](https://mlsanity.com/paper/1906.06423/) (2019)
- [What makes ImageNet good for transfer learning?](https://mlsanity.com/paper/1608.08614/) (2016)
- [The Devil is in the Tails: Fine-grained Classification in the Wild](https://mlsanity.com/paper/1709.01450/) (2017)
- [Metric Learning with Adaptive Density Discrimination](https://mlsanity.com/paper/1511.05939/) (2015)
- [Metric Learning With HORDE: High-Order Regularizer for Deep Embeddings](https://mlsanity.com/paper/1908.02735/) (2019)
- [Unsupervised Deep Metric Learning with Transformed Attention Consistency and Contrastive Clustering Loss](https://mlsanity.com/paper/2008.04378/) (2020)
- [Semi-supervised semantic segmentation needs strong, varied perturbations](https://mlsanity.com/paper/1906.01916/) (2019)
- [Improving Prostate Cancer Detection with Breast Histopathology Images](https://mlsanity.com/paper/1903.05769/) (2019)

## Abstract (arXiv)

Jitendra Malik once said, "Supervision is the opium of the AI researcher". Most deep learning techniques heavily rely on extreme amounts of human labels to work effectively. In today's world, the rate of data creation greatly surpasses the rate of data annotation. Full reliance on human annotations is just a temporary means to solve current closed problems in AI. In reality, only a tiny fraction of data is annotated. Annotation Efficient Learning (AEL) is a study of algorithms to train models effectively with fewer annotations. To thrive in AEL environments, we need deep learning techniques that rely less on manual annotations (e.g., image, bounding-box, and per-pixel labels), but learn useful information from unlabeled data. In this thesis, we explore five different techniques for handling AEL.