# Task Decomposition for Efficient Annotation

arXiv: https://arxiv.org/abs/2606.24734 · HTML: https://mlsanity.com/paper/2606.24734/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 67 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding](https://mlsanity.com/paper/2401.04398/) (2024)
- [Break It Down: A Question Understanding Benchmark](https://mlsanity.com/paper/2001.11770/) (2020)
- [Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples!](https://mlsanity.com/paper/2303.08559/) (2023)
- [GoLLIE: Annotation Guidelines improve Zero-Shot Information-Extraction](https://mlsanity.com/paper/2310.03668/) (2023)
- [Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks](https://mlsanity.com/paper/2410.20266/) (2024)
- [CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation](https://mlsanity.com/paper/2310.15638/) (2023)
- [Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction](https://mlsanity.com/paper/2303.04132/) (2023)
- [LLMs Accelerate Annotation for Medical Information Extraction](https://mlsanity.com/paper/2312.02296/) (2023)
- [Large-Scale QA-SRL Parsing](https://mlsanity.com/paper/1805.05377/) (2018)
- [NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data](https://mlsanity.com/paper/2402.15343/) (2024)
- [Sequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks](https://mlsanity.com/paper/1906.03783/) (2019)
- [Syntactic Scaffolds for Semantic Structures](https://mlsanity.com/paper/1808.10485/) (2018)
- [Unsupervised Label-aware Event Trigger and Argument Classification](https://mlsanity.com/paper/2012.15243/) (2020)
- [Are Large Language Models Robust Coreference Resolvers?](https://mlsanity.com/paper/2305.14489/) (2023)
- [Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement](https://mlsanity.com/paper/2502.06207/) (2025)
- [Re-TASK: Revisiting LLM Tasks from Capability, Skill, and Knowledge Perspectives](https://mlsanity.com/paper/2408.06904/) (2024)
- [Learning Joint Semantic Parsers from Disjoint Data](https://mlsanity.com/paper/1804.05990/) (2018)
- [Deep Multitask Learning for Semantic Dependency Parsing](https://mlsanity.com/paper/1704.06855/) (2017)
- [Predicting Annotation Difficulty to Improve Task Routing and Model Performance for Biomedical Information Extraction](https://mlsanity.com/paper/1905.07791/) (2019)
- [Major Entity Identification: A Generalizable Alternative to Coreference Resolution](https://mlsanity.com/paper/2406.14654/) (2024)
- [Information Extraction from Visually Rich Documents using LLM-based Organization of Documents into Independent Textual Segments](https://mlsanity.com/paper/2505.13535/) (2025)
- [Mention Annotations Alone Enable Efficient Domain Adaptation for Coreference Resolution](https://mlsanity.com/paper/2210.07602/) (2022)
- [ComplexityNet: Increasing LLM Inference Efficiency by Learning Task Complexity](https://mlsanity.com/paper/2312.11511/) (2023)
- [Familiarity: Better Evaluation of Zero-Shot Named Entity Recognition by Quantifying Label Shifts in Synthetic Training Data](https://mlsanity.com/paper/2412.10121/) (2024)
- [Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training](https://mlsanity.com/paper/2305.12634/) (2023)
- [The Case for Scalable, Data-Driven Theory: A Paradigm for Scientific Progress in NLP](https://mlsanity.com/paper/2312.00349/) (2023)

## Abstract (arXiv)

High-quality annotations of structured representations are expensive to collect over large corpora. Manual annotation of structure is laborious, and model-based annotation, although cheaper to generate, requires expensive validation and potentially significant supervision to ensure that the annotation quality is strong enough to be useful downstream. In traditional annotation workflows, annotation of each complete example is performed end-to-end by a single annotator. However, structured annotation is complex, and each aspect of the task represents a unique challenge with an associated inferential load for a given annotator. Modern annotation projects can incorporate heterogeneous groups of annotators, including both models and human annotators with varying domain and linguistic expertise. It remains unclear, however, how to redesign annotation tasks in this setting, where efforts are discriminately allocated across heterogeneous annotators with respect to distinct annotation challenges. We propose to decompose annotation tasks into sub-tasks in order to reduce the aggregate inferential load of annotation projects. Inspired by the notion of centers from centering theory, we introduce a formal model of inferential load based on the degrees of freedom in the space of valid annotations. Using this model, we show that identifying these centers (i.e. salient anchor entities realized by annotation sub-tasks) constrains the output space complexity, and decompositions which isolate and advance center identification reduce the aggregate inferential load. We provide guidelines for decomposing complex structured annotation tasks, supported by examples demonstrating improved cost-efficiency from our prior work. Finally, we present a procedure for allocating sub-tasks across annotators to maximize quality under a fixed budget.