# Reasoning-Enhanced Domain-Adaptive Pretraining of Multimodal Large Language Models for Short Video Content Governance

arXiv: https://arxiv.org/abs/2509.21486 · HTML: https://mlsanity.com/paper/2509.21486/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)

## Builds on (most cited first, 16 of 16)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification](https://mlsanity.com/paper/2508.21046/) (2025)
- [Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2502.10708/) (2025)
- [Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy](https://mlsanity.com/paper/2402.07270/) (2024)
- [Efficient Continual Pre-training by Mitigating the Stability Gap](https://mlsanity.com/paper/2406.14833/) (2024)
- [VLM as Policy: Common-Law Content Moderation Framework for Short Video Platform](https://mlsanity.com/paper/2504.14904/) (2025)
- [Audio-Enhanced Vision-Language Modeling with Latent Space Broadening for High Quality Data Expansion](https://mlsanity.com/paper/2503.17551/) (2025)
- [Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training](https://mlsanity.com/paper/2411.14318/) (2024)
- [Learning to segment fetal brain tissue from noisy annotations](https://mlsanity.com/paper/2203.14962/) (2022)
- [IPS: In-Prompt Process Supervision for Short Video Content Moderation](https://mlsanity.com/paper/2412.15251/) (2024)
- [Structure-aware Domain Knowledge Injection for Large Language Models](https://mlsanity.com/paper/2407.16724/) (2024)
- [Revisiting the Message Passing in Heterophilous Graph Neural Networks](https://mlsanity.com/paper/2405.17768/) (2024)
- [Embedding-based Retrieval in Multimodal Content Moderation](https://mlsanity.com/paper/2507.01066/) (2025)
- [Improved Few-Shot Image Classification Through Multiple-Choice Questions](https://mlsanity.com/paper/2407.16145/) (2024)

## Built on by (most cited first, 16 of 16)

- [Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views](https://mlsanity.com/paper/2510.18632/) (2025)
- [RoboRouter: Training-Free Policy Routing for Robotic Manipulation](https://mlsanity.com/paper/2603.07892/) (2026)
- [The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Management](https://mlsanity.com/paper/2605.23071/) (2026)
- [Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation](https://mlsanity.com/paper/2604.06950/) (2026)
- [Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization](https://mlsanity.com/paper/2607.08057/) (2026)
- [MIS-HCC: Hierarchical Channel Clustering for Efficient Medical Image Segmentation](https://mlsanity.com/paper/2607.17329/) (2026)
- [FAST: A Synergistic Framework of Attention and State-space Models for Spatiotemporal Traffic Prediction](https://mlsanity.com/paper/2604.13453/) (2026)
- [MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Models](https://mlsanity.com/paper/2608.22236/) (2026)
- [When Rules Fall Short: Agent-Driven Discovery of Emerging Content Issues in Short Video Platforms](https://mlsanity.com/paper/2601.11634/) (2026)
- [Outsmarting the Chameleon: Counterfactual Decoupling for Tactical OOD Shifts in Live Streaming Risk Assessment](https://mlsanity.com/paper/2606.02946/) (2026)
- [Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignment](https://mlsanity.com/paper/2511.08399/) (2025)
- [UNIVID: Unified Vision-Language Model for Video Moderation](https://mlsanity.com/paper/2606.05748/) (2026)
- [Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents](https://mlsanity.com/paper/2609.33875/) (2026)
- [Second-Order Problem Solving for Recursive Self-Improvement in Formal Verification](https://mlsanity.com/paper/2610.05701/) (2026)
- [Static Gradient Attribution Underperforms a Density-Matched Random Mask on Loss-Based Forgetting Within LoRA B-Matrix](https://mlsanity.com/paper/2610.12744/) (2026)
- [Beyond Semantic IDs: Finite-Beam Reachability and Continuous-Query Generation in Generative Recommendation](https://mlsanity.com/paper/2610.12977/) (2026)

## Abstract (arXiv)

Short video platforms are evolving rapidly, making the identification of inappropriate content increasingly critical. Existing approaches typically train separate and small classification models for each type of issue, which requires extensive human-labeled data and lacks cross-issue generalization. We propose a reasoning-enhanced multimodal large language model (MLLM) pretraining paradigm for unified inappropriate content detection. To address the distribution gap between short video content and the original pretraining data of MLLMs, as well as the complex issue definitions, we introduce three targeted pretraining tasks: (1) textitCaption, to enhance the MLLM's perception of video details; (2) textitVisual Question Answering (VQA), to deepen the MLLM's understanding of issue definitions and annotation guidelines; (3) textitChain-of-Thought (CoT), to enhance the MLLM's reasoning capability. Experimental results show that our pretraining approach significantly improves the MLLM's performance in both zero-shot and supervised fine-tuning (SFT) settings. In addition, our pretrained model demonstrates strong generalization capabilities to emergent, previously unseen issues.