# Perceptual Adversarial Networks for Image-to-Image Transformation

arXiv: https://arxiv.org/abs/1706.09138 · HTML: https://mlsanity.com/paper/1706.09138/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2017.

## Built on by (most cited first, 30 of 42)

- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers](https://mlsanity.com/paper/2111.12710/) (2021)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Recent Advances for Quantum Neural Networks in Generative Learning](https://mlsanity.com/paper/2206.03066/) (2022)
- [Semi-Supervised Video Deraining with Dynamical Rain Generator](https://mlsanity.com/paper/2103.07939/) (2021)
- [Fair Generative Models via Transfer Learning](https://mlsanity.com/paper/2212.00926/) (2022)
- [GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields](https://mlsanity.com/paper/2306.06044/) (2023)
- [RORem: Training a Robust Object Remover with Human-in-the-Loop](https://mlsanity.com/paper/2501.00740/) (2025)
- [Cross-Modality Deep Feature Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2201.02356/) (2022)
- [A Comprehensive Survey on Data-Efficient GANs in Image Generation](https://mlsanity.com/paper/2204.08329/) (2022)
- [AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection](https://mlsanity.com/paper/2106.05499/) (2021)
- [FakeCLR: Exploring Contrastive Learning for Solving Latent Discontinuity in Data-Efficient GANs](https://mlsanity.com/paper/2207.08630/) (2022)
- [Generative AI for Physical Layer Communications: A Survey](https://mlsanity.com/paper/2312.05594/) (2023)
- [Uncertainty-Guided Progressive GANs for Medical Image Translation](https://mlsanity.com/paper/2106.15542/) (2021)
- [Adversarial Score Distillation: When score distillation meets GAN](https://mlsanity.com/paper/2312.00739/) (2023)
- [UniCode: Learning a Unified Codebook for Multimodal Large Language Models](https://mlsanity.com/paper/2403.09072/) (2024)
- [DeViT: Deformed Vision Transformers in Video Inpainting](https://mlsanity.com/paper/2209.13925/) (2022)
- [Contrastive Feature Loss for Image Prediction](https://mlsanity.com/paper/2111.06934/) (2021)
- [Exploring The Effect of High-frequency Components in GANs Training](https://mlsanity.com/paper/2103.11093/) (2021)
- [AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection](https://mlsanity.com/paper/2310.13103/) (2023)
- [Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining](https://mlsanity.com/paper/2505.16811/) (2025)
- [PeaceGAN: A GAN-based Multi-Task Learning Method for SAR Target Image Generation with a Pose Estimator and an Auxiliary Classifier](https://mlsanity.com/paper/2103.15469/) (2021)
- [Efficient High-Resolution Image-to-Image Translation using Multi-Scale Gradient U-Net](https://mlsanity.com/paper/2105.13067/) (2021)
- [Stochastic Actor-Executor-Critic for Image-to-Image Translation](https://mlsanity.com/paper/2112.07403/) (2021)
- [Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?](https://mlsanity.com/paper/2203.11141/) (2022)
- [Efficient CNN-based Super Resolution Algorithms for mmWave Mobile Radar Imaging](https://mlsanity.com/paper/2305.02092/) (2023)
- [H-GAN: the power of GANs in your Hands](https://mlsanity.com/paper/2103.15017/) (2021)
- [OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System](https://mlsanity.com/paper/2303.00501/) (2023)
- [High-dimensional Assisted Generative Model for Color Image Restoration](https://mlsanity.com/paper/2108.06460/) (2021)
- [Training Generative Adversarial Networks with Adaptive Composite Gradient](https://mlsanity.com/paper/2111.05508/) (2021)

## Abstract (arXiv)

In this paper, we propose a principled Perceptual Adversarial Networks (PAN) for image-to-image transformation tasks. Unlike existing application-specific algorithms, PAN provides a generic framework of learning mapping relationship between paired images (Fig. 1), such as mapping a rainy image to its de-rained counterpart, object edges to its photo, semantic labels to a scenes image, etc. The proposed PAN consists of two feed-forward convolutional neural networks (CNNs), the image transformation network T and the discriminative network D. Through combining the generative adversarial loss and the proposed perceptual adversarial loss, these two networks can be trained alternately to solve image-to-image transformation tasks. Among them, the hidden layers and output of the discriminative network D are upgraded to continually and automatically discover the discrepancy between the transformed image and the corresponding ground-truth. Simultaneously, the image transformation network T is trained to minimize the discrepancy explored by the discriminative network D. Through the adversarial training process, the image transformation network T will continually narrow the gap between transformed images and ground-truth images. Experiments evaluated on several image-to-image transformation tasks (e.g., image de-raining, image inpainting, etc.) show that the proposed PAN outperforms many related state-of-the-art methods.