# Best Practices for Managing Data Annotation Projects

arXiv: https://arxiv.org/abs/2009.11654 · HTML: https://mlsanity.com/paper/2009.11654/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2020.

## Built on by (most cited first, 12 of 12)

- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Experimental Standards for Deep Learning in Natural Language Processing Research](https://mlsanity.com/paper/2204.06251/) (2022)
- [MCRanker: Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankers](https://mlsanity.com/paper/2404.11960/) (2024)
- [LLMs as Span Annotators: A Comparative Study of LLMs and Humans](https://mlsanity.com/paper/2504.08697/) (2025)
- [Automatic Labelling with Open-source LLMs using Dynamic Label Schema Integration](https://mlsanity.com/paper/2501.12332/) (2025)
- [Hands-On Tutorial: Labeling with LLM and Human-in-the-Loop](https://mlsanity.com/paper/2411.04637/) (2024)
- [Overcoming Language Barriers: Multilingual Analysis of the 2023 Swiss Privacy Law's Impact](https://mlsanity.com/paper/2510.05860/) (2025)
- [CAPC-CG: A Large-Scale, Expert-Directed LLM-Annotated Corpus of Adaptive Policy Communication in China](https://mlsanity.com/paper/2510.08986/) (2025)
- [We Need to Talk About Data: The Importance of Data Readiness in Natural Language Processing](https://mlsanity.com/paper/2110.05464/) (2021)
- [Generalizable Error Modeling for Human Data Annotation: Evidence From an Industry-Scale Search Data Annotation Program](https://mlsanity.com/paper/2310.05286/) (2023)
- [Abusive Span Detection for Vietnamese Narrative Texts](https://mlsanity.com/paper/2312.07831/) (2023)
- [Designing A Sustainable Marine Debris Clean-up Framework without Human Labels](https://mlsanity.com/paper/2405.14815/) (2024)

## Abstract (arXiv)

Annotation is the labeling of data by human effort. Annotation is critical to modern machine learning, and Bloomberg has developed years of experience of annotation at scale. This report captures a wealth of wisdom for applied annotation projects, collected from more than 30 experienced annotation project managers in Bloomberg's Global Data department.