# PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration

arXiv: https://arxiv.org/abs/2007.12142 · HTML: https://mlsanity.com/paper/2007.12142/

## Key facts

- Cited by 178 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2020.

## Built on by (most cited first, 30 of 178)

- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild](https://mlsanity.com/paper/2401.13627/) (2024)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution](https://mlsanity.com/paper/2501.11561/) (2025)
- [Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution](https://mlsanity.com/paper/2203.09195/) (2022)
- [Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models](https://mlsanity.com/paper/2312.08962/) (2023)
- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [Q-Insight: Understanding Image Quality via Visual Reinforcement Learning](https://mlsanity.com/paper/2503.22679/) (2025)
- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillation](https://mlsanity.com/paper/2404.01717/) (2024)
- [Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network](https://mlsanity.com/paper/2204.10485/) (2022)
- [Masked Image Training for Generalizable Deep Image Denoising](https://mlsanity.com/paper/2303.13132/) (2023)
- [Perceptual Image Quality Assessment with Transformers](https://mlsanity.com/paper/2104.14730/) (2021)
- [DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution](https://mlsanity.com/paper/2503.23580/) (2025)
- [Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streaks](https://mlsanity.com/paper/2308.14153/) (2023)
- [Shift-tolerant Perceptual Similarity Metric](https://mlsanity.com/paper/2207.13686/) (2022)
- [RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18035/) (2024)
- [Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment](https://mlsanity.com/paper/2108.07948/) (2021)
- [Reflash Dropout in Image Super-Resolution](https://mlsanity.com/paper/2112.12089/) (2021)
- [NTIRE 2022 Challenge on Perceptual Image Quality Assessment](https://mlsanity.com/paper/2206.11695/) (2022)
- [Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Dataset](https://mlsanity.com/paper/2405.18842/) (2024)
- [Harnessing Diffusion-Yielded Score Priors for Image Restoration](https://mlsanity.com/paper/2507.20590/) (2025)
- [Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumption](https://mlsanity.com/paper/2503.11221/) (2025)
- [PyTorch Image Quality: Metrics for Image Quality Assessment](https://mlsanity.com/paper/2208.14818/) (2022)
- [Quality-Aware Image-Text Alignment for Opinion-Unaware Image Quality Assessment](https://mlsanity.com/paper/2403.11176/) (2024)
- [Lossy Compression with Pretrained Diffusion Models](https://mlsanity.com/paper/2501.09815/) (2025)
- [Exploring the Naturalness of AI-Generated Images](https://mlsanity.com/paper/2312.05476/) (2023)

## Abstract (arXiv)

Image quality assessment (IQA) is the key factor for the fast development of image restoration (IR) algorithms. The most recent IR methods based on Generative Adversarial Networks (GANs) have achieved significant improvement in visual performance, but also presented great challenges for quantitative evaluation. Notably, we observe an increasing inconsistency between perceptual quality and the evaluation results. Then we raise two questions: (1) Can existing IQA methods objectively evaluate recent IR algorithms? (2) When focus on beating current benchmarks, are we getting better IR algorithms? To answer these questions and promote the development of IQA methods, we contribute a large-scale IQA dataset, called Perceptual Image Processing Algorithms (PIPAL) dataset. Especially, this dataset includes the results of GAN-based methods, which are missing in previous datasets. We collect more than 1.13 million human judgments to assign subjective scores for PIPAL images using the more reliable "Elo system". Based on PIPAL, we present new benchmarks for both IQA and super-resolution methods. Our results indicate that existing IQA methods cannot fairly evaluate GAN-based IR algorithms. While using appropriate evaluation methods is important, IQA methods should also be updated along with the development of IR algorithms. At last, we improve the performance of IQA networks on GAN-based distortions by introducing anti-aliasing pooling. Experiments show the effectiveness of the proposed method.