# Triple-level Model Inferred Collaborative Network Architecture for Video Deraining

arXiv: https://arxiv.org/abs/2111.04459 · HTML: https://mlsanity.com/paper/2111.04459/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 66 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 24 of 24)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Video Super-resolution with Temporal Group Attention](https://mlsanity.com/paper/2007.10595/) (2020)
- [A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton](https://mlsanity.com/paper/2006.04045/) (2020)
- [Single Image Deraining: From Model-Based to Data-Driven and Beyond](https://mlsanity.com/paper/1912.07150/) (2019)
- [A First Order Method for Solving Convex Bi-Level Optimization Problems](https://mlsanity.com/paper/1702.03999/) (2017)
- [Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond](https://mlsanity.com/paper/2110.00455/) (2021)
- [Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework](https://mlsanity.com/paper/1803.10433/) (2018)
- [FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors](https://mlsanity.com/paper/1803.07487/) (2018)
- [Learning Temporal Coherence via Self-Supervision for GAN-based Video Generation](https://mlsanity.com/paper/1811.09393/) (2018)
- [Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising](https://mlsanity.com/paper/1909.08228/) (2019)
- [Investigating Task-driven Latent Feasibility for Nonconvex Image Modeling](https://mlsanity.com/paper/1910.08242/) (2019)
- [Removing Rain in Videos: A Large-scale Database and A Two-stream ConvLSTM Approach](https://mlsanity.com/paper/1906.02526/) (2019)
- [Structural Residual Learning for Single Image Rain Removal](https://mlsanity.com/paper/2005.09228/) (2020)
- [Investigating Customization Strategies and Convergence Behaviors of Task-specific ADMM](https://mlsanity.com/paper/1909.10819/) (2019)

## Built on by (most cited first, 9 of 9)

- [A Generalized Physical-knowledge-guided Dynamic Model for Underwater Image Enhancement](https://mlsanity.com/paper/2308.05447/) (2023)
- [EGVD: Event-Guided Video Deraining](https://mlsanity.com/paper/2309.17239/) (2023)
- [An End-to-End Cascaded Image Deraining and Object Detection Neural Network](https://mlsanity.com/paper/2202.11279/) (2022)
- [Relationship Quantification of Image Degradations](https://mlsanity.com/paper/2212.04148/) (2022)
- [Little Strokes Fell Great Oaks: Boosting the Hierarchical Features for Multi-exposure Image Fusion](https://mlsanity.com/paper/2404.06033/) (2024)
- [Histogram-guided Video Colorization Structure with Spatial-Temporal Connection](https://mlsanity.com/paper/2308.04899/) (2023)
- [Towards General and Fast Video Derain via Knowledge Distillation](https://mlsanity.com/paper/2308.05346/) (2023)
- [RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics](https://mlsanity.com/paper/2607.13802/) (2026)
- [Learning with Bilevel-Minimax Optimization for Efficient and Reliable Transfer Attacks](https://mlsanity.com/paper/2608.11815/) (2026)

## Abstract (arXiv)

Video deraining is an important issue for outdoor vision systems and has been investigated extensively. However, designing optimal architectures by the aggregating model formation and data distribution is a challenging task for video deraining. In this paper, we develop a model-guided triple-level optimization framework to deduce network architecture with cooperating optimization and auto-searching mechanism, named Triple-level Model Inferred Cooperating Searching (TMICS), for dealing with various video rain circumstances. In particular, to mitigate the problem that existing methods cannot cover various rain streaks distribution, we first design a hyper-parameter optimization model about task variable and hyper-parameter. Based on the proposed optimization model, we design a collaborative structure for video deraining. This structure includes Dominant Network Architecture (DNA) and Companionate Network Architecture (CNA) that is cooperated by introducing an Attention-based Averaging Scheme (AAS). To better explore inter-frame information from videos, we introduce a macroscopic structure searching scheme that searches from Optical Flow Module (OFM) and Temporal Grouping Module (TGM) to help restore latent frame. In addition, we apply the differentiable neural architecture searching from a compact candidate set of task-specific operations to discover desirable rain streaks removal architectures automatically. Extensive experiments on various datasets demonstrate that our model shows significant improvements in fidelity and temporal consistency over the state-of-the-art works. Source code is available at https://github.com/vis-opt-group/TMICS.