# propella-1: Multi-Property Document Annotation for LLM Data Curation at Scale

arXiv: https://arxiv.org/abs/2602.12414 · HTML: https://mlsanity.com/paper/2602.12414/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 71 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Builds on (most cited first, 23 of 23)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [DataComp-LM: In search of the next generation of training sets for language models](https://mlsanity.com/paper/2406.11794/) (2024)
- [RedPajama: an Open Dataset for Training Large Language Models](https://mlsanity.com/paper/2411.12372/) (2024)
- [Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset](https://mlsanity.com/paper/2412.02595/) (2024)
- [QuRating: Selecting High-Quality Data for Training Language Models](https://mlsanity.com/paper/2402.09739/) (2024)
- [FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language](https://mlsanity.com/paper/2506.20920/) (2025)
- [Enhancing Multilingual LLM Pretraining with Model-Based Data Selection](https://mlsanity.com/paper/2502.10361/) (2025)
- [How Reliable is Multilingual LLM-as-a-Judge?](https://mlsanity.com/paper/2505.12201/) (2025)
- [BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining](https://mlsanity.com/paper/2508.10975/) (2025)
- [Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models](https://mlsanity.com/paper/2504.14194/) (2025)
- [DataMan: Data Manager for Pre-training Large Language Models](https://mlsanity.com/paper/2502.19363/) (2025)
- [Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models](https://mlsanity.com/paper/2505.22232/) (2025)
- [HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models](https://mlsanity.com/paper/2511.01066/) (2025)
- [Register Always Matters: Analysis of LLM Pretraining Data Through the Lens of Language Variation](https://mlsanity.com/paper/2504.01542/) (2025)
- [The German Commons - 154 Billion Tokens of Openly Licensed Text for German Language Models](https://mlsanity.com/paper/2510.13996/) (2025)

## Built on by (most cited first, 6 of 6)

- [ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance](https://mlsanity.com/paper/2604.12378/) (2026)
- [Mechanistic Circuit Identification for Controllable Data Generation](https://mlsanity.com/paper/2608.24065/) (2026)
- [Laguna M.1/XS.2 Technical Report](https://mlsanity.com/paper/2605.27605/) (2026)
- [The Announcement Carries the Cue: Markup, Boundaries, and the Notation of Pre-Training Corpora](https://mlsanity.com/paper/2608.09093/) (2026)
- [It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs](https://mlsanity.com/paper/2609.37891/) (2026)
- [Scaling Laws for Multimodal Data Mixtures](https://mlsanity.com/paper/2610.14423/) (2026)

## Abstract (arXiv)

Since FineWeb-Edu, data curation for LLM pretraining has predominantly relied on single scalar quality scores produced by small classifiers. A single score conflates multiple quality dimensions, prevents flexible filtering, and offers no interpretability. We introduce propella-1, a family of small multilingual LLMs (0.6B, 1.7B, 4B parameters) that annotate text documents across 18 properties organized into six categories: core content, classification, quality and value, audience and purpose, safety and compliance, and geographic relevance. The models support 57 languages and produce structured JSON annotations conforming to a predefined schema. Evaluated against a frontier commercial LLM as a reference annotator, the 4B model achieves higher agreement than much larger general-purpose models. We release propella-annotations, a dataset of over three billion document annotations covering major pretraining corpora including data from FineWeb-2, FinePDFs, HPLT 3.0, and Nemotron-CC. Using these annotations, we present a multi-dimensional compositional analysis of widely used pretraining datasets, revealing substantial differences in quality, reasoning depth, and content composition that single-score approaches cannot capture. All model weights and annotations are released under permissive, commercial-use licenses.