# CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes

arXiv: https://arxiv.org/abs/2606.31435 · HTML: https://mlsanity.com/paper/2606.31435/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Builds on (most cited first, 28 of 28)

- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Efficient Attentions for Long Document Summarization](https://mlsanity.com/paper/2104.02112/) (2021)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [HelpSteer2: Open-source dataset for training top-performing reward models](https://mlsanity.com/paper/2406.08673/) (2024)
- [Fine-grained Hallucination Detection and Editing for Language Models](https://mlsanity.com/paper/2401.06855/) (2024)
- [Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails](https://mlsanity.com/paper/2501.09004/) (2025)
- [FaithDial: A Faithful Benchmark for Information-Seeking Dialogue](https://mlsanity.com/paper/2204.10757/) (2022)
- [The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://mlsanity.com/paper/2202.00443/) (2022)
- [MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction](https://mlsanity.com/paper/2211.07342/) (2022)
- [Scaling Laws of Synthetic Data for Language Models](https://mlsanity.com/paper/2503.19551/) (2025)
- [Large Language Models as Data Preprocessors](https://mlsanity.com/paper/2308.16361/) (2023)
- [KramaBench: A Benchmark for AI Systems on Data-to-Insight Pipelines over Data Lakes](https://mlsanity.com/paper/2506.06541/) (2025)
- [EditEval: An Instruction-Based Benchmark for Text Improvements](https://mlsanity.com/paper/2209.13331/) (2022)
- [AutoDCWorkflow: LLM-based Data Cleaning Workflow Auto-Generation and Benchmark](https://mlsanity.com/paper/2412.06724/) (2024)
- [DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle](https://mlsanity.com/paper/2512.04324/) (2025)
- [Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications](https://mlsanity.com/paper/2502.13358/) (2025)
- [LLMClean: Context-Aware Tabular Data Cleaning via LLM-Generated OFDs](https://mlsanity.com/paper/2404.18681/) (2024)
- [Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models](https://mlsanity.com/paper/2501.14755/) (2024)
- [A Survey on Data Cleaning Methods for Improved Machine Learning Model Performance](https://mlsanity.com/paper/2109.07127/) (2021)
- [PRvL: Quantifying the Capabilities and Risks of Large Language Models for PII Redaction](https://mlsanity.com/paper/2508.05545/) (2025)
- [XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updates](https://mlsanity.com/paper/2309.11063/) (2023)
- [DCA-Bench: A Benchmark for Dataset Curation Agents](https://mlsanity.com/paper/2406.07275/) (2024)
- [The Empirical Impact of Data Sanitization on Language Models](https://mlsanity.com/paper/2411.05978/) (2024)
- [ChatPipe: Orchestrating Data Preparation Program by Optimizing Human-ChatGPT Interactions](https://mlsanity.com/paper/2304.03540/) (2023)
- [Tackling the Inherent Difficulty of Noise Filtering in RAG](https://mlsanity.com/paper/2601.01896/) (2026)
- [Preprocessing Methods and Pipelines of Data Mining: An Overview](https://mlsanity.com/paper/1906.08510/) (2019)

## Built on by (most cited first, 1 of 1)

- [ArrivalBench: Agent-Generated Data Pipelines Are Correct Once and Wrong Under Time](https://mlsanity.com/paper/2610.02363/) (2026)

## Abstract (arXiv)

Data refinement involves executing multi-step recipes over evolving text states, where both composition and execution order of processing operators determine the outcome. While existing benchmarks either isolate text editing or entangle it with code and tool execution, it remains unclear whether LLMs can directly and faithfully execute these compositional, order-sensitive data refinement recipes. To fill this gap, we introduce CDR-Bench, a comprehensive benchmark featuring 3,462 high-quality tasks spanning four real-world data refinement domains and 29 distinct operators. Our benchmark evaluates models across atomic, order-agnostic, and order-sensitive settings, leveraging deterministic reference outputs to enable exact evaluation. Experiments on 10+ state-of-the-art LLMs reveal consistent failure patterns: performance degrades sharply in compositional settings, and order-sensitive recipe success collapses. These findings underline that current LLMs lack the procedural faithfulness required for reliable compositional data refinement.