# Scaling Up LLM Reviews for Google Ads Content Moderation

arXiv: https://arxiv.org/abs/2402.14590 · HTML: https://mlsanity.com/paper/2402.14590/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 5 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 4 of 4)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [PaLI-X: On Scaling up a Multilingual Vision and Language Model](https://mlsanity.com/paper/2305.18565/) (2023)

## Built on by (most cited first, 12 of 12)

- [PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs](https://mlsanity.com/paper/2409.14729/) (2024)
- [ControlNET: A Firewall for RAG-based LLM System](https://mlsanity.com/paper/2504.09593/) (2025)
- [Real-World Gaps in AI Governance Research](https://mlsanity.com/paper/2505.00174/) (2025)
- [RAVEN++: Pinpointing Fine-Grained Violations in Advertisement Videos with Active Reinforcement Reasoning](https://mlsanity.com/paper/2511.19168/) (2025)
- [Scaling Reinforcement Learning for Content Moderation with Large Language Models](https://mlsanity.com/paper/2512.20061/) (2025)
- [Gen-AI for User Safety: A Survey](https://mlsanity.com/paper/2411.06606/) (2024)
- [Zero-Shot Image Moderation in Google Ads with LLM-Assisted Textual Descriptions and Cross-modal Co-embeddings](https://mlsanity.com/paper/2412.16215/) (2024)
- [Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond](https://mlsanity.com/paper/2410.18114/) (2024)
- [Transforming User Defined Criteria into Explainable Indicators with an Integrated LLM AHP System](https://mlsanity.com/paper/2601.05267/) (2025)
- [Provably Secure Retrieval-Augmented Generation](https://mlsanity.com/paper/2508.01084/) (2025)
- [Agile Deliberation: Concept Deliberation for Subjective Visual Classification](https://mlsanity.com/paper/2512.10821/) (2025)
- [Summarize, Judge, Refine: Decoupled Content Understanding and Policy Learning for Multimodal Content Moderation](https://mlsanity.com/paper/2609.22094/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are powerful tools for content moderation, but their inference costs and latency make them prohibitive for casual use on large datasets, such as the Google Ads repository. This study proposes a method for scaling up LLM reviews for content moderation in Google Ads. First, we use heuristics to select candidates via filtering and duplicate removal, and create clusters of ads for which we select one representative ad per cluster. We then use LLMs to review only the representative ads. Finally, we propagate the LLM decisions for the representative ads back to their clusters. This method reduces the number of reviews by more than 3 orders of magnitude while achieving a 2x recall compared to a baseline non-LLM model. The success of this approach is a strong function of the representations used in clustering and label propagation; we found that cross-modal similarity representations yield better results than uni-modal representations.