# Leveraging Large-scale Multimedia Datasets to Refine Content Moderation Models

arXiv: https://arxiv.org/abs/2212.00668 · HTML: https://mlsanity.com/paper/2212.00668/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 10 of 10)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Reliable Decision from Multiple Subtasks through Threshold Optimization: Content Moderation in the Wild](https://mlsanity.com/paper/2208.07522/) (2022)
- [Applying deep learning to classify pornographic images and videos](https://mlsanity.com/paper/1511.08899/) (2015)
- [On-Device Content Moderation](https://mlsanity.com/paper/2107.11845/) (2021)
- [TAR on Social Media: A Framework for Online Content Moderation](https://mlsanity.com/paper/2108.12752/) (2021)

## Built on by (most cited first, 4 of 4)

- [Disturbing Image Detection Using LMM-Elicited Emotion Embeddings](https://mlsanity.com/paper/2406.12668/) (2024)
- [Bias in the Loop: How Humans Evaluate AI-Generated Suggestions](https://mlsanity.com/paper/2509.08514/) (2025)
- [Mitigating Viewer Impact from Disturbing Imagery using AI Filters: A User-Study](https://mlsanity.com/paper/2307.10334/) (2023)
- [VModA: An Effective Framework for Adaptive NSFW Image Moderation](https://mlsanity.com/paper/2505.23386/) (2025)

## Abstract (arXiv)

The sheer volume of online user-generated content has rendered content moderation technologies essential in order to protect digital platform audiences from content that may cause anxiety, worry, or concern. Despite the efforts towards developing automated solutions to tackle this problem, creating accurate models remains challenging due to the lack of adequate task-specific training data. The fact that manually annotating such data is a highly demanding procedure that could severely affect the annotators' emotional well-being is directly related to the latter limitation. In this paper, we propose the CM-Refinery framework that leverages large-scale multimedia datasets to automatically extend initial training datasets with hard examples that can refine content moderation models, while significantly reducing the involvement of human annotators. We apply our method on two model adaptation strategies designed with respect to the different challenges observed while collecting data, i.e. lack of (i) task-specific negative data or (ii) both positive and negative data. Additionally, we introduce a diversity criterion applied to the data collection process that further enhances the generalization performance of the refined models. The proposed method is evaluated on the Not Safe for Work (NSFW) and disturbing content detection tasks on benchmark datasets achieving 1.32% and 1.94% accuracy improvements compared to the state of the art, respectively. Finally, it significantly reduces human involvement, as 92.54% of data are automatically annotated in case of disturbing content while no human intervention is required for the NSFW task.