# Data-Juicer 2.0: Cloud-Scale Adaptive Data Processing for and with Foundation Models

arXiv: https://arxiv.org/abs/2501.14755 · HTML: https://mlsanity.com/paper/2501.14755/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 72 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 30 of 30)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [Emu3: Next-Token Prediction is All You Need](https://mlsanity.com/paper/2409.18869/) (2024)
- [The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only](https://mlsanity.com/paper/2306.01116/) (2023)
- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation](https://mlsanity.com/paper/2307.06942/) (2023)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models](https://mlsanity.com/paper/2403.18814/) (2024)
- [WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Research](https://mlsanity.com/paper/2303.17395/) (2023)
- [Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text](https://mlsanity.com/paper/2304.06939/) (2023)
- [A Survey of Multimodal Large Language Model from A Data-centric Perspective](https://mlsanity.com/paper/2405.16640/) (2024)
- [Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes](https://mlsanity.com/paper/2312.06353/) (2023)
- [Data-Juicer: A One-Stop Data Processing System for Large Language Models](https://mlsanity.com/paper/2309.02033/) (2023)
- [Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and Benchmarks](https://mlsanity.com/paper/2306.04362/) (2023)
- [Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Models](https://mlsanity.com/paper/2408.04594/) (2024)
- [The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective](https://mlsanity.com/paper/2407.08583/) (2024)
- [Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development](https://mlsanity.com/paper/2407.11784/) (2024)

## Built on by (most cited first, 12 of 12)

- [Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Data](https://mlsanity.com/paper/2502.04380/) (2025)
- [FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents](https://mlsanity.com/paper/2603.01712/) (2026)
- [MindGYM: What Matters in Question Synthesis for Thinking-Centric Fine-Tuning?](https://mlsanity.com/paper/2503.09499/) (2025)
- [DetailMaster: Can Your Text-to-Image Model Handle Long Prompts?](https://mlsanity.com/paper/2505.16915/) (2025)
- [SciDataCopilot: An Agentic Data Preparation Framework for AGI-driven Scientific Discovery](https://mlsanity.com/paper/2602.09132/) (2026)
- [Grounded in Reality: Learning and Deploying Proactive LLM from Offline Logs](https://mlsanity.com/paper/2510.25441/) (2025)
- [DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving](https://mlsanity.com/paper/2606.07001/) (2026)
- [Learning from Reasoning Failures via Synthetic Data Generation](https://mlsanity.com/paper/2504.14523/) (2025)
- [CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes](https://mlsanity.com/paper/2606.31435/) (2026)
- [VeriSciQA: An Auto-Verified Dataset for Scientific Visual Question Answering](https://mlsanity.com/paper/2511.19899/) (2025)
- [STREAM: A Data-Centric Framework for Mining High-Value Task-Oriented Dialogues from Streaming Media](https://mlsanity.com/paper/2605.25162/) (2026)
- [VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes](https://mlsanity.com/paper/2609.06652/) (2026)

## Abstract (arXiv)

Foundation models demand advanced data processing for their vast, multimodal datasets. However, traditional frameworks struggle with the unique complexities of multimodal data. In response, we present Data-Juicer 2.0, a data processing system backed by 100+ data processing operators spanning text, image, video, and audio modalities, supporting more critical tasks including data analysis, synthesis, annotation, and foundation model post-training. With seamless compatibility and dedicated optimization for popular dataset hubs like Hugging Face and computing engines like Ray, it improves upon its predecessor in terms of usability, efficiency, and programmability. It features an easily accessible user interface layer that supports decoupled Python interactions, RESTful APIs, and conversational commands. Its new runtime layer offers adaptive execution across diverse scales and environments, abstracting away system complexities. Extensive empirical evaluations demonstrate Data-Juicer 2.0's remarkable performance and scalability, highlighting its capability to efficiently process TB-level data with 10k+ CPU cores. The system is publicly available and has been widely adopted in diverse research fields and real-world products such as Alibaba Cloud PAI. We actively maintain the system and share practical insights to foster research and applications of next-generation foundation models.