# Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1804.03312 · HTML: https://mlsanity.com/paper/1804.03312/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2018.

## Built on by (most cited first, 30 of 47)

- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [ClassSR: A General Framework to Accelerate Super-Resolution Networks by Data Characteristic](https://mlsanity.com/paper/2103.04039/) (2021)
- [TAPE: Task-Agnostic Prior Embedding for Image Restoration](https://mlsanity.com/paper/2203.06074/) (2022)
- [NTIRE 2021 Challenge on Image Deblurring](https://mlsanity.com/paper/2104.14854/) (2021)
- [RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18035/) (2024)
- [An Intelligent Agentic System for Complex Image Restoration Problems](https://mlsanity.com/paper/2410.17809/) (2024)
- [Bringing Old Films Back to Life](https://mlsanity.com/paper/2203.17276/) (2022)
- [ReLLIE: Deep Reinforcement Learning for Customized Low-Light Image Enhancement](https://mlsanity.com/paper/2107.05830/) (2021)
- [Learning Distortion Invariant Representation for Image Restoration from A Causality Perspective](https://mlsanity.com/paper/2303.06859/) (2023)
- [Toward Real-World Super-Resolution via Adaptive Downsampling Models](https://mlsanity.com/paper/2109.03444/) (2021)
- [Pik-Fix: Restoring and Colorizing Old Photos](https://mlsanity.com/paper/2205.01902/) (2022)
- [Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy](https://mlsanity.com/paper/2210.07598/) (2022)
- [Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network](https://mlsanity.com/paper/2009.14563/) (2020)
- [R3L: Connecting Deep Reinforcement Learning to Recurrent Neural Networks for Image Denoising via Residual Recovery](https://mlsanity.com/paper/2107.05318/) (2021)
- [DRL-ISP: Multi-Objective Camera ISP with Deep Reinforcement Learning](https://mlsanity.com/paper/2207.03081/) (2022)
- [Spatial-temporal Hierarchical Reinforcement Learning for Interpretable Pathology Image Super-Resolution](https://mlsanity.com/paper/2406.18310/) (2024)
- [OPERA: An Agent for Image Restoration with End-to-End Joint Planning-Execution Optimization](https://mlsanity.com/paper/2605.22104/) (2026)
- [Simulating analogue film damage to analyse and improve artefact restoration on high-resolution scans](https://mlsanity.com/paper/2302.10004/) (2023)
- [ALL-E: Aesthetics-guided Low-light Image Enhancement](https://mlsanity.com/paper/2304.14610/) (2023)
- [Position: Agentic Systems Constitute a Key Component of Next-Generation Intelligent Image Processing](https://mlsanity.com/paper/2505.16007/) (2025)
- [TooBadRL: Trigger Optimization to Boost Effectiveness of Backdoor Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/2506.09562/) (2025)
- [Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration](https://mlsanity.com/paper/2601.19506/) (2026)
- [Sparsely ensembled convolutional neural network classifiers via reinforcement learning](https://mlsanity.com/paper/2102.03921/) (2021)
- [REPNP: Plug-and-Play with Deep Reinforcement Learning Prior for Robust Image Restoration](https://mlsanity.com/paper/2207.12056/) (2022)
- [RL-AWB: Deep Reinforcement Learning for Auto White Balance Correction in Low-Light Night-time Scenes](https://mlsanity.com/paper/2601.05249/) (2026)
- [Adaptive Remote Sensing Image Attribute Learning for Active Object Detection](https://mlsanity.com/paper/2101.06438/) (2021)
- [A Reinforcement Learning Approach for Sequential Spatial Transformer Networks](https://mlsanity.com/paper/2106.14295/) (2021)

## Abstract (arXiv)

We investigate a novel approach for image restoration by reinforcement learning. Unlike existing studies that mostly train a single large network for a specialized task, we prepare a toolbox consisting of small-scale convolutional networks of different complexities and specialized in different tasks. Our method, RL-Restore, then learns a policy to select appropriate tools from the toolbox to progressively restore the quality of a corrupted image. We formulate a step-wise reward function proportional to how well the image is restored at each step to learn the action policy. We also devise a joint learning scheme to train the agent and tools for better performance in handling uncertainty. In comparison to conventional human-designed networks, RL-Restore is capable of restoring images corrupted with complex and unknown distortions in a more parameter-efficient manner using the dynamically formed toolchain.