# Multi-Scale Representation Learning for Image Restoration with State-Space Model

arXiv: https://arxiv.org/abs/2408.10145 · HTML: https://mlsanity.com/paper/2408.10145/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 47 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Kindling the Darkness: A Practical Low-light Image Enhancer](https://mlsanity.com/paper/1905.04161/) (2019)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Learning Enriched Features for Real Image Restoration and Enhancement](https://mlsanity.com/paper/2003.06792/) (2020)
- [Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement](https://mlsanity.com/paper/2012.05609/) (2020)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [A Model-driven Deep Neural Network for Single Image Rain Removal](https://mlsanity.com/paper/2005.01333/) (2020)
- [Structure-Preserving Deraining with Residue Channel Prior Guidance](https://mlsanity.com/paper/2108.09079/) (2021)
- [FreqMamba: Viewing Mamba from a Frequency Perspective for Image Deraining](https://mlsanity.com/paper/2404.09476/) (2024)
- [Dynamic Attentive Graph Learning for Image Restoration](https://mlsanity.com/paper/2109.06620/) (2021)
- [Lightweight Adaptive Feature De-drifting for Compressed Image Classification](https://mlsanity.com/paper/2401.01724/) (2024)
- [MPRNet: Multi-Path Residual Network for Lightweight Image Super Resolution](https://mlsanity.com/paper/2011.04566/) (2020)
- [Mutual Information-driven Triple Interaction Network for Efficient Image Dehazing](https://mlsanity.com/paper/2308.06998/) (2023)
- [Double Domain Guided Real-Time Low-Light Image Enhancement for Ultra-High-Definition Transportation Surveillance](https://mlsanity.com/paper/2309.08382/) (2023)
- [Dual Residual Attention Network for Image Denoising](https://mlsanity.com/paper/2305.04269/) (2023)

## Built on by (most cited first, 15 of 15)

- [VasTSD: Learning 3D Vascular Tree-state Space Diffusion Model for Angiography Synthesis](https://mlsanity.com/paper/2503.12758/) (2025)
- [Depth-Synergized Mamba Meets Memory Experts for All-Day Image Reflection Separation](https://mlsanity.com/paper/2601.00322/) (2026)
- [Diffusion-Based Image Editing: An Unforeseen Adversary to Robust Invisible Watermarks](https://mlsanity.com/paper/2511.05598/) (2025)
- [SSCM: A Spatial-Semantic Consistent Model for Multi-Contrast MRI Super-Resolution](https://mlsanity.com/paper/2509.18593/) (2025)
- [Vanishing Watermarks: Diffusion-Based Image Editing Undermines Robust Invisible Watermarking](https://mlsanity.com/paper/2602.20680/) (2026)
- [DiffVQA: Video Quality Assessment Using Diffusion Feature Extractor](https://mlsanity.com/paper/2505.03261/) (2025)
- [Editing on the Generative Manifold: A Theoretical and Empirical Study of General Diffusion-Based Image Editing Trade-offs](https://mlsanity.com/paper/2603.29736/) (2026)
- [EmambaIR: Efficient Visual State Space Model for Event-guided Image Reconstruction](https://mlsanity.com/paper/2605.08073/) (2026)
- [SemaMIL: Semantic-Aware Multiple Instance Learning with Retrieval-Guided State Space Modeling for Whole Slide Images](https://mlsanity.com/paper/2509.00442/) (2025)
- [Robust Concept Erasure in Diffusion Models: A Theoretical Perspective on Security and Robustness](https://mlsanity.com/paper/2509.12024/) (2025)
- [DeRainMamba: A Frequency-Aware State Space Model with Detail Enhancement for Image Deraining](https://mlsanity.com/paper/2510.06746/) (2025)
- [When Denoising Becomes Unsigning: Theoretical and Empirical Analysis of Watermark Fragility Under Diffusion-Based Image Editing](https://mlsanity.com/paper/2603.04696/) (2026)
- [Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarking](https://mlsanity.com/paper/2603.12949/) (2026)
- [ReconMIL: Synergizing Latent Space Reconstruction with Bi-Stream Mamba for Whole Slide Image Analysis](https://mlsanity.com/paper/2603.19925/) (2026)
- [Interactive State Space Model with Cross-Modal Local Scanning for Depth Super-Resolution](https://mlsanity.com/paper/2605.11934/) (2026)

## Abstract (arXiv)

Image restoration endeavors to reconstruct a high-quality, detail-rich image from a degraded counterpart, which is a pivotal process in photography and various computer vision systems. In real-world scenarios, different types of degradation can cause the loss of image details at various scales and degrade image contrast. Existing methods predominantly rely on CNN and Transformer to capture multi-scale representations. However, these methods are often limited by the high computational complexity of Transformers and the constrained receptive field of CNN, which hinder them from achieving superior performance and efficiency in image restoration. To address these challenges, we propose a novel Multi-Scale State-Space Model-based (MS-Mamba) for efficient image restoration that enhances the capacity for multi-scale representation learning through our proposed global and regional SSM modules. Additionally, an Adaptive Gradient Block (AGB) and a Residual Fourier Block (RFB) are proposed to improve the network's detail extraction capabilities by capturing gradients in various directions and facilitating learning details in the frequency domain. Extensive experiments on nine public benchmarks across four classic image restoration tasks, image deraining, dehazing, denoising, and low-light enhancement, demonstrate that our proposed method achieves new state-of-the-art performance while maintaining low computational complexity. The source code will be publicly available.