# DMOps: Data Management Operation and Recipes

arXiv: https://arxiv.org/abs/2301.01228 · HTML: https://mlsanity.com/paper/2301.01228/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 2 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective](https://mlsanity.com/paper/1811.03402/) (2018)
- [Neural Machine Translation: A Review and Survey](https://mlsanity.com/paper/1912.02047/) (2019)
- [TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task](https://mlsanity.com/paper/2004.14855/) (2020)
- [DataPerf: Benchmarks for Data-Centric AI Development](https://mlsanity.com/paper/2207.10062/) (2022)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [Generate, Annotate, and Learn: NLP with Synthetic Text](https://mlsanity.com/paper/2106.06168/) (2021)
- [What can Data-Centric AI Learn from Data and ML Engineering?](https://mlsanity.com/paper/2112.06439/) (2021)
- [LightTag: Text Annotation Platform](https://mlsanity.com/paper/2109.02320/) (2021)
- [A Large-Scale Dataset for Benchmarking Elevator Button Segmentation and Character Recognition](https://mlsanity.com/paper/2103.09030/) (2021)
- [How should human translation coexist with NMT? Efficient tool for building high quality parallel corpus](https://mlsanity.com/paper/2111.00191/) (2021)
- [Enterprise AI Canvas -- Integrating Artificial Intelligence into Business](https://mlsanity.com/paper/2009.11190/) (2020)
- [Can Artificial Intelligence Transform DevOps?](https://mlsanity.com/paper/2206.00225/) (2022)

## Built on by (most cited first, 7 of 7)

- [Advancing Post-OCR Correction: A Comparative Study of Synthetic Data](https://mlsanity.com/paper/2408.02253/) (2024)
- [Inter-Annotator Agreement in the Wild: Uncovering Its Emerging Roles and Considerations in Real-World Scenarios](https://mlsanity.com/paper/2306.14373/) (2023)
- [Transcending Traditional Boundaries: Leveraging Inter-Annotator Agreement (IAA) for Enhancing Data Management Operations (DMOps)](https://mlsanity.com/paper/2306.14374/) (2023)
- [Model-Based Data-Centric AI: Bridging the Divide Between Academic Ideals and Industrial Pragmatism](https://mlsanity.com/paper/2403.01832/) (2024)
- [Dataverse: Open-Source ETL (Extract, Transform, Load) Pipeline for Large Language Models](https://mlsanity.com/paper/2403.19340/) (2024)
- [Synthetic Alone: Exploring the Dark Side of Synthetic Data for Grammatical Error Correction](https://mlsanity.com/paper/2306.14377/) (2023)
- [Modeling User-System Behavior for Training-free Building of Private Domain Conversational Agents](https://mlsanity.com/paper/2511.10283/) (2025)

## Abstract (arXiv)

Data-centric AI has shed light on the significance of data within the machine learning (ML) pipeline. Recognizing its significance, academia, industry, and government departments have suggested various NLP data research initiatives. While the ability to utilize existing data is essential, the ability to build a dataset has become more critical than ever, especially in the industry. In consideration of this trend, we propose a "Data Management Operations and Recipes" to guide the industry in optimizing the building of datasets for NLP products. This paper presents the concept of DMOps which is derived from real-world experiences with NLP data management and aims to streamline data operations by offering a baseline.