# AnomalyXFusion: Multi-modal Anomaly Synthesis with Diffusion

arXiv: https://arxiv.org/abs/2404.19444 · HTML: https://mlsanity.com/paper/2404.19444/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [Differentiable Augmentation for Data-Efficient GAN Training](https://mlsanity.com/paper/2006.10738/) (2020)
- [WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation](https://mlsanity.com/paper/2303.14814/) (2023)
- [Few-shot Image Generation via Cross-domain Correspondence](https://mlsanity.com/paper/2104.06820/) (2021)
- [Deep Industrial Image Anomaly Detection: A Survey](https://mlsanity.com/paper/2301.11514/) (2023)
- [Multimodal Industrial Anomaly Detection via Hybrid Fusion](https://mlsanity.com/paper/2303.00601/) (2023)
- [CFA: Coupled-hypersphere-based Feature Adaptation for Target-Oriented Anomaly Localization](https://mlsanity.com/paper/2206.04325/) (2022)
- [AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model](https://mlsanity.com/paper/2312.05767/) (2023)
- [Real3D-AD: A Dataset of Point Cloud Anomaly Detection](https://mlsanity.com/paper/2309.13226/) (2023)
- [Omni-frequency Channel-selection Representations for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2203.00259/) (2022)
- [StyleGAN2 Distillation for Feed-forward Image Manipulation](https://mlsanity.com/paper/2003.03581/) (2020)
- [Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection](https://mlsanity.com/paper/2103.15158/) (2021)
- [Pushing the Limits of Fewshot Anomaly Detection in Industry Vision: Graphcore](https://mlsanity.com/paper/2301.12082/) (2023)
- [MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities](https://mlsanity.com/paper/2205.00908/) (2022)
- [Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation](https://mlsanity.com/paper/2303.02389/) (2023)
- [IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing](https://mlsanity.com/paper/2301.13359/) (2023)
- [Segment Any Anomaly without Training via Hybrid Prompt Regularization](https://mlsanity.com/paper/2305.10724/) (2023)
- [Prototypical Residual Networks for Anomaly Detection and Localization](https://mlsanity.com/paper/2212.02031/) (2022)
- [ISTR: End-to-End Instance Segmentation with Transformers](https://mlsanity.com/paper/2105.00637/) (2021)
- [EasyNet: An Easy Network for 3D Industrial Anomaly Detection](https://mlsanity.com/paper/2307.13925/) (2023)

## Built on by (most cited first, 21 of 21)

- [Unseen Visual Anomaly Generation](https://mlsanity.com/paper/2406.01078/) (2024)
- [One-for-More: Continual Diffusion Model for Anomaly Detection](https://mlsanity.com/paper/2502.19848/) (2025)
- [A Survey on Diffusion Models for Anomaly Detection](https://mlsanity.com/paper/2501.11430/) (2025)
- [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025)
- [AnomalyControl: Learning Cross-modal Semantic Features for Controllable Anomaly Synthesis](https://mlsanity.com/paper/2412.06510/) (2024)
- [A Survey on Industrial Anomaly Synthesis](https://mlsanity.com/paper/2502.16412/) (2025)
- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [AnomalyPainter: Vision-Language-Diffusion Synergy for Zero-Shot Realistic and Diverse Industrial Anomaly Synthesis](https://mlsanity.com/paper/2503.07253/) (2025)
- ["Stones from Other Hills can Polish Jade": Zero-shot Anomaly Image Synthesis via Cross-domain Anomaly Injection](https://mlsanity.com/paper/2501.15211/) (2025)
- [Generate Aligned Anomaly: Region-Guided Few-Shot Anomaly Image-Mask Pair Synthesis for Industrial Inspection](https://mlsanity.com/paper/2507.09619/) (2025)
- [3D-PNAS: 3D Industrial Surface Anomaly Synthesis with Perlin Noise](https://mlsanity.com/paper/2504.12856/) (2025)
- [Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era](https://mlsanity.com/paper/2511.00540/) (2025)
- [One-to-More: High-Fidelity Training-Free Anomaly Generation with Attention Control](https://mlsanity.com/paper/2603.18093/) (2026)
- [Anomaly Detection by Effectively Leveraging Synthetic Images](https://mlsanity.com/paper/2512.23227/) (2025)
- [Training-Free Anomaly Generation via Dual-Attention Enhancement in Diffusion Model](https://mlsanity.com/paper/2508.11550/) (2025)
- [FAST: Foreground-aware Diffusion with Accelerated Sampling Trajectory for Segmentation-oriented Anomaly Synthesis](https://mlsanity.com/paper/2509.20295/) (2025)
- [Test-Time Adaptation for Anomaly Segmentation via Topology-Aware Optimal Transport Chaining](https://mlsanity.com/paper/2601.20333/) (2026)
- [SteelDefectX: A Multi-Form Vision-Language Dataset and Benchmark for Steel Surface Defect Analysis](https://mlsanity.com/paper/2603.21824/) (2026)
- [Learning Topology-Aware Representations via Test-Time Adaptation for Anomaly Segmentation](https://mlsanity.com/paper/2606.28268/) (2026)
- [OSAGEN: Object-Aware Mask Priors and Multistage Decoupled Diffusion for Industrial Anomaly Generation](https://mlsanity.com/paper/2607.29533/) (2026)
- [Multi-Modal Anomaly Detection: A Survey](https://mlsanity.com/paper/2608.24937/) (2026)

## Abstract (arXiv)

Anomaly synthesis is one of the effective methods to augment abnormal samples for training. However, current anomaly synthesis methods predominantly rely on texture information as input, which limits the fidelity of synthesized abnormal samples. Because texture information is insufficient to correctly depict the pattern of anomalies, especially for logical anomalies. To surmount this obstacle, we present the AnomalyXFusion framework, designed to harness multi-modality information to enhance the quality of synthesized abnormal samples. The AnomalyXFusion framework comprises two distinct yet synergistic modules: the Multi-modal In-Fusion (MIF) module and the Dynamic Dif-Fusion (DDF) module. The MIF module refines modality alignment by aggregating and integrating various modality features into a unified embedding space, termed X-embedding, which includes image, text, and mask features. Concurrently, the DDF module facilitates controlled generation through an adaptive adjustment of X-embedding conditioned on the diffusion steps. In addition, to reveal the multi-modality representational power of AnomalyXFusion, we propose a new dataset, called MVTec Caption. More precisely, MVTec Caption extends 2.2k accurate image-mask-text annotations for the MVTec AD and LOCO datasets. Comprehensive evaluations demonstrate the effectiveness of AnomalyXFusion, especially regarding the fidelity and diversity for logical anomalies. Project page: http:github.com/hujiecpp/MVTec-Caption