# ASAP: Advancing Semantic Alignment Promotes Multi-Modal Manipulation Detecting and Grounding

arXiv: https://arxiv.org/abs/2412.12718 · HTML: https://mlsanity.com/paper/2412.12718/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2024.

## Built on by (most cited first, 15 of 15)

- [Multi-speaker Attention Alignment for Multimodal Social Interaction](https://mlsanity.com/paper/2511.17952/) (2025)
- [The Coherence Trap: When MLLM-Crafted Narratives Exploit Manipulated Visual Contexts](https://mlsanity.com/paper/2505.17476/) (2025)
- [FakeSV-VLM: Taming VLM for Detecting Fake Short-Video News via Progressive Mixture-Of-Experts Adapter](https://mlsanity.com/paper/2508.19639/) (2025)
- [Beyond Artificial Misalignment: Detecting and Grounding Semantic-Coordinated Multimodal Manipulations](https://mlsanity.com/paper/2509.12653/) (2025)
- [Towards Unified Multimodal Misinformation Detection in Social Media: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2509.25991/) (2025)
- [Fine-grained Multiple Supervisory Network for Multi-modal Manipulation Detecting and Grounding](https://mlsanity.com/paper/2508.02479/) (2025)
- [A Spatial-Frequency Aware Multi-Scale Fusion Network for Real-Time Deepfake Detection](https://mlsanity.com/paper/2508.20449/) (2025)
- [DGM4+: Dataset Extension for Global Scene Inconsistency](https://mlsanity.com/paper/2509.26047/) (2025)
- [Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection](https://mlsanity.com/paper/2603.01993/) (2026)
- [CIEC: Coupling Implicit and Explicit Cues for Multimodal Weakly Supervised Manipulation Localization](https://mlsanity.com/paper/2602.02175/) (2026)
- [REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection](https://mlsanity.com/paper/2605.28459/) (2026)
- [SGS: Segmentation-Guided Scoring for Global Scene Inconsistencies](https://mlsanity.com/paper/2509.26039/) (2025)
- [Insight-A: Attribution-aware for Multimodal Misinformation Detection](https://mlsanity.com/paper/2511.21705/) (2025)
- [Bridging Pixels and Words: Mask-Aware Local Semantic Fusion for Multimodal Media Verification](https://mlsanity.com/paper/2603.26052/) (2026)
- [Lightweight Stylistic Consistency Profiling: Robust Detection of LLM-Generated Textual Content for Multimedia Moderation](https://mlsanity.com/paper/2605.05950/) (2026)

## Abstract (arXiv)

We present ASAP, a new framework for detecting and grounding multi-modal media manipulation (DGM4).Upon thorough examination, we observe that accurate fine-grained cross-modal semantic alignment between the image and text is vital for accurately manipulation detection and grounding. While existing DGM4 methods pay rare attention to the cross-modal alignment, hampering the accuracy of manipulation detecting to step further. To remedy this issue, this work targets to advance the semantic alignment learning to promote this task. Particularly, we utilize the off-the-shelf Multimodal Large-Language Models (MLLMs) and Large Language Models (LLMs) to construct paired image-text pairs, especially for the manipulated instances. Subsequently, a cross-modal alignment learning is performed to enhance the semantic alignment. Besides the explicit auxiliary clues, we further design a Manipulation-Guided Cross Attention (MGCA) to provide implicit guidance for augmenting the manipulation perceiving. With the grounding truth available during training, MGCA encourages the model to concentrate more on manipulated components while downplaying normal ones, enhancing the model's ability to capture manipulations. Extensive experiments are conducted on the DGM4 dataset, the results demonstrate that our model can surpass the comparison method with a clear margin.