# The DALPHI annotation framework & how its pre-annotations can improve annotator efficiency

arXiv: https://arxiv.org/abs/1808.05558 · HTML: https://mlsanity.com/paper/1808.05558/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2018.

## Built on by (most cited first, 1 of 1)

- [A Domain-Specific Curated Benchmark for Entity and Document-Level Relation Extraction](https://mlsanity.com/paper/2602.04320/) (2026)

## Abstract (arXiv)

Producing the required amounts of training data for machine learning and NLP tasks often involves human annotators doing very repetitive and monotonous work. In this paper, we present and evaluate our novel annotation framework DALPHI, which facilitates the annotation process by providing the annotator with suggestions generated by an automated, active-learning based assistance system. In a study with 66 participants, we demonstrate on the exemplary task of annotating named entities in text documents that with this assistance system the annotation processes can be improved with respect to the quality and quantity of produced annotations, even if the pre-annotations provided by the assistance system are at a recall level of only 50%.