# A Guide for Manual Annotation of Scientific Imagery: How to Prepare for Large Projects

arXiv: https://arxiv.org/abs/2508.14801 · HTML: https://mlsanity.com/paper/2508.14801/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 57 references that are in the map.
- First posted on arXiv 20 Aug 2025.

## Builds on (most cited first, 8 of 8)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets](https://mlsanity.com/paper/2104.12690/) (2021)
- [Multiscale IoU: A Metric for Evaluation of Salient Object Detection with Fine Structures](https://mlsanity.com/paper/2105.14572/) (2021)
- [A Curated Image Parameter Dataset from Solar Dynamics Observatory Mission](https://mlsanity.com/paper/1906.01062/) (2019)

## Abstract (arXiv)

Despite the high demand for manually annotated image data, managing complex and costly annotation projects remains under-discussed. This is partly due to the fact that leading such projects requires dealing with a set of diverse and interconnected challenges which often fall outside the expertise of specific domain experts, leaving practical guidelines scarce. These challenges range widely from data collection to resource allocation and recruitment, from mitigation of biases to effective training of the annotators. This paper provides a domain-agnostic preparation guide for annotation projects, with a focus on scientific imagery. Drawing from the authors' extensive experience in managing a large manual annotation project, it addresses fundamental concepts including success measures, annotation subjects, project goals, data availability, and essential team roles. Additionally, it discusses various human biases and recommends tools and technologies to improve annotation quality and efficiency. The goal is to encourage further research and frameworks for creating a comprehensive knowledge base to reduce the costs of manual annotation projects across various fields.