# RU-AI: A Large Multimodal Dataset for Machine-Generated Content Detection

arXiv: https://arxiv.org/abs/2406.04906 · HTML: https://mlsanity.com/paper/2406.04906/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 64 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 30 of 33)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment](https://mlsanity.com/paper/2310.01852/) (2023)
- [GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image](https://mlsanity.com/paper/2306.08571/) (2023)
- [YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone](https://mlsanity.com/paper/2112.02418/) (2021)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [Automatic Detection of Machine Generated Text: A Critical Survey](https://mlsanity.com/paper/2011.01314/) (2020)
- [Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis](https://mlsanity.com/paper/2404.13686/) (2024)
- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis](https://mlsanity.com/paper/2103.05630/) (2021)
- [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023)
- [TweepFake: about Detecting Deepfake Tweets](https://mlsanity.com/paper/2008.00036/) (2020)
- [Regulating ChatGPT and other Large Generative AI Models](https://mlsanity.com/paper/2302.02337/) (2023)
- [Detecting and Grounding Multi-Modal Media Manipulation](https://mlsanity.com/paper/2304.02556/) (2023)
- [CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms](https://mlsanity.com/paper/1706.07068/) (2017)
- [Art and the science of generative AI: A deeper dive](https://mlsanity.com/paper/2306.04141/) (2023)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [Deep Multimodal Semantic Embeddings for Speech and Images](https://mlsanity.com/paper/1511.03690/) (2015)
- [Text-Free Image-to-Speech Synthesis Using Learned Segmental Units](https://mlsanity.com/paper/2012.15454/) (2020)
- [Learning Word-Like Units from Joint Audio-Visual Analysis](https://mlsanity.com/paper/1701.07481/) (2017)

## Built on by (most cited first, 4 of 4)

- [Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.08826/) (2025)
- [Could AI Trace and Explain the Origins of AI-Generated Images and Text?](https://mlsanity.com/paper/2504.04279/) (2025)
- [From Generation to Detection: A Multimodal Multi-Task Dataset for Benchmarking Health Misinformation](https://mlsanity.com/paper/2505.18685/) (2025)
- [NAU-QMUL: Utilizing BERT and CLIP for Multi-modal AI-Generated Image Detection](https://mlsanity.com/paper/2602.23863/) (2026)

## Abstract (arXiv)

The recent generative AI models' capability of creating realistic and human-like content is significantly transforming the ways in which people communicate, create and work. The machine-generated content is a double-edged sword. On one hand, it can benefit the society when used appropriately. On the other hand, it may mislead people, posing threats to the society, especially when mixed together with natural content created by humans. Hence, there is an urgent need to develop effective methods to detect machine-generated content. However, the lack of aligned multimodal datasets inhibited the development of such methods, particularly in triple-modality settings (e.g., text, image, and voice). In this paper, we introduce RU-AI, a new large-scale multimodal dataset for robust and effective detection of machine-generated content in text, image and voice. Our dataset is constructed on the basis of three large publicly available datasets: Flickr8K, COCO and Places205, by adding their corresponding AI duplicates, resulting in a total of 1,475,370 instances. In addition, we created an additional noise variant of the dataset for testing the robustness of detection models. We conducted extensive experiments with the current SOTA detection methods on our dataset. The results reveal that existing models still struggle to achieve accurate and robust detection on our dataset. We hope that this new data set can promote research in the field of machine-generated content detection, fostering the responsible use of generative AI. The source code and datasets are available at https://github.com/ZhihaoZhang97/RU-AI.