# FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restoration

arXiv: https://arxiv.org/abs/2412.01427 · HTML: https://mlsanity.com/paper/2412.01427/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 69 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 30 of 49)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild](https://mlsanity.com/paper/2401.13627/) (2024)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [DiffIR: Efficient Diffusion Model for Image Restoration](https://mlsanity.com/paper/2303.09472/) (2023)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters](https://mlsanity.com/paper/2403.11549/) (2024)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [Image Restoration with Mean-Reverting Stochastic Differential Equations](https://mlsanity.com/paper/2301.11699/) (2023)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [InstructIR: High-Quality Image Restoration Following Human Instructions](https://mlsanity.com/paper/2401.16468/) (2024)
- [Deep Generalized Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2204.13348/) (2022)
- [Incremental Few-Shot Object Detection](https://mlsanity.com/paper/2003.04668/) (2020)
- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [LEDNet: Joint Low-light Enhancement and Deblurring in the Dark](https://mlsanity.com/paper/2202.03373/) (2022)
- [OneRestore: A Universal Restoration Framework for Composite Degradation](https://mlsanity.com/paper/2407.04621/) (2024)
- [Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement](https://mlsanity.com/paper/2302.11831/) (2023)

## Built on by (most cited first, 30 of 39)

- [DiT4SR: Taming Diffusion Transformer for Real-World Image Super-Resolution](https://mlsanity.com/paper/2503.23580/) (2025)
- [Learning Continuous Wasserstein Barycenter Space for Generalized All-in-One Image Restoration](https://mlsanity.com/paper/2602.23169/) (2026)
- [UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity](https://mlsanity.com/paper/2412.20157/) (2024)
- [ClearAIR: A Human-Visual-Perception-Inspired All-in-One Image Restoration](https://mlsanity.com/paper/2601.02763/) (2026)
- [FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Model](https://mlsanity.com/paper/2512.09282/) (2025)
- [RealRestorer: Towards Generalizable Real-World Image Restoration with Large-Scale Image Editing Models](https://mlsanity.com/paper/2603.25502/) (2026)
- [Bilevel Layer-Positioning LoRA for Real Image Dehazing](https://mlsanity.com/paper/2603.10872/) (2026)
- [DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restoration](https://mlsanity.com/paper/2504.21487/) (2025)
- [Rethinking Nighttime Image Deraining via Learnable Color Space Transformation](https://mlsanity.com/paper/2510.17440/) (2025)
- [UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancement](https://mlsanity.com/paper/2512.06750/) (2025)
- [LoViF 2026 Challenge on Real-World All-in-One Image Restoration: Methods and Results](https://mlsanity.com/paper/2604.19445/) (2026)
- [Decoupled Residual Denoising Diffusion Models for Unified and Data Efficient Image-to-Image Translation](https://mlsanity.com/paper/2606.01048/) (2026)
- [Universal Image Restoration Pre-training via Masked Degradation Classification](https://mlsanity.com/paper/2510.13282/) (2025)
- [FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image Restoration](https://mlsanity.com/paper/2511.14099/) (2025)
- [TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration](https://mlsanity.com/paper/2601.20306/) (2026)
- [Towards Unified Image Deblurring using a Mixture-of-Experts Decoder](https://mlsanity.com/paper/2508.06228/) (2025)
- [V-Bridge: Bridging Video Generative Priors to Versatile Few-shot Image Restoration](https://mlsanity.com/paper/2603.13089/) (2026)
- [LoViF 2026 The First Challenge on Weather Removal in Videos](https://mlsanity.com/paper/2604.10655/) (2026)
- [EndoIR: Degradation-Agnostic All-in-One Endoscopic Image Restoration via Noise-Aware Routing Diffusion](https://mlsanity.com/paper/2511.05873/) (2025)
- [UniRain: Unified Image Deraining with RAG-based Dataset Distillation and Multi-objective Reweighted Optimization](https://mlsanity.com/paper/2603.03967/) (2026)
- [X-Restormer++: 1st Place Solution for the UG2+ CVPR 2026 All-Weather Restoration Challenge](https://mlsanity.com/paper/2605.13258/) (2026)
- [PixRestore: Unified Image Restoration via Pixel Diffusion Transformer](https://mlsanity.com/paper/2608.16793/) (2026)
- [Degradation-Aware Metric Prompting for Hyperspectral Image Restoration](https://mlsanity.com/paper/2512.20251/) (2025)
- [Unifying Heterogeneous Degradations: Uncertainty-Aware Diffusion Bridge Model for All-in-One Image Restoration](https://mlsanity.com/paper/2601.21592/) (2026)
- [Leveraging pretrained RGB denoisers for hyperspectral image restoration](https://mlsanity.com/paper/2605.24769/) (2026)
- [GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration](https://mlsanity.com/paper/2605.31039/) (2026)
- [The Second LoViF 2026 Challenge on Real-World All-in-One Image Restoration: Methods and Results](https://mlsanity.com/paper/2607.21118/) (2026)
- [Bridging Degradation Discrimination and Generation for Universal Image Restoration](https://mlsanity.com/paper/2602.00579/) (2026)
- [Your Pre-trained Diffusion Model Secretly Knows Restoration](https://mlsanity.com/paper/2604.04924/) (2026)
- [SmartPhotoCrafter: Unified Reasoning, Generation and Optimization for Automatic Photographic Image Editing](https://mlsanity.com/paper/2604.19587/) (2026)

## Abstract (arXiv)

Despite the significant progress made by all-in-one models in universal image restoration, existing methods suffer from a generalization bottleneck in real-world scenarios, as they are mostly trained on small-scale synthetic datasets with limited degradations. Therefore, large-scale high-quality real-world training data is urgently needed to facilitate the emergence of foundational models for image restoration. To advance this field, we spare no effort in contributing a million-scale dataset with two notable advantages over existing training data: real-world samples with larger-scale, and degradation types with higher diversity. By adjusting internal camera settings and external imaging conditions, we can capture aligned image pairs using our well-designed data acquisition system over multiple rounds and our data alignment criterion. Moreover, we propose a robust model, FoundIR, to better address a broader range of restoration tasks in real-world scenarios, taking a further step toward foundation models. Specifically, we first utilize a diffusion-based generalist model to remove degradations by learning the degradation-agnostic common representations from diverse inputs, where incremental learning strategy is adopted to better guide model training. To refine the model's restoration capability in complex scenarios, we introduce degradation-aware specialist models for achieving final high-quality results. Extensive experiments show the value of our dataset and the effectiveness of our method.