mlsanityData to 7 Oct 2026

Total Variation Optimization Layers for Computer Vision

arXiv 2204.03643 ↗First posted 7 Apr 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Optimization within a layer of a deep-net has emerged as a new direction for deep-net layer design. However, there are two main challenges when applying these layers to computer vision tasks: (a) which optimization problem within a layer is useful?; (b) how to ensure that computation within a layer remains efficient? To study question (a), in this work, we propose total variation (TV) minimization as a layer for computer vision. Motivated by the success of total variation in image processing, we hypothesize that TV as a layer provides useful inductive bias for deep-nets too. We study this hypothesis on five computer vision tasks: image classification, weakly supervised object localization, edge-preserving smoothing, edge detection, and image denoising, improving over existing baselines. To achieve these results we had to address question (b): we developed a GPU-based projected-Newton method which is 37× faster than existing solutions.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 46 of its 88 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

46 of 88 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  11. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  12. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
34 more
  1. 2018Group Normalizationcited by 1,514
  2. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  3. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  4. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  5. 2019Deep Equilibrium Modelscited by 603
  6. 2015Holistically-Nested Edge Detectioncited by 590
  7. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  8. 2019Differentiable Convex Optimization Layerscited by 458
  9. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 426
  10. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  11. 2019Differentiation of Blackbox Combinatorial Solverscited by 224
  12. 2017Video Frame Synthesis using Deep Voxel Flowcited by 209
  13. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  14. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  15. 2016Semantic Image Inpainting with Deep Generative Modelscited by 156
  16. 2020Multiscale Deep Equilibrium Modelscited by 149
  17. 2016Richer Convolutional Features for Edge Detectioncited by 141
  18. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 121
  19. 2019Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detectioncited by 104
  20. 2020Evaluating Weakly Supervised Object Localization Methods Rightcited by 104
  21. 2020Deep Graph Matching via Blackbox Differentiation of Combinatorial Solverscited by 81
  22. 2019Deep Declarative Networks: A New Hopecited by 77
  23. 2015Structured Prediction Energy Networkscited by 59
  24. 2019Optimizing Rank-based Metrics with Blackbox Differentiationcited by 58
  25. 2020Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learningcited by 53
  26. 2019Chirality Nets for Human Pose Regressioncited by 31
  27. 20213D Spatial Recognition without Spatially Labeled 3Dcited by 31
  28. 2018Interpretable and Globally Optimal Prediction for Textual Grounding using Image Conceptscited by 15
  29. 2018Deep Structured Prediction with Nonlinear Output Transformationscited by 12
  30. 2014Modular proximal optimization for multidimensional total-variation regularizationcited by 10
  31. 2019Graph Structured Prediction Energy Networkscited by 10
  32. 2020Learning to solve TV regularized problems with unrolled algorithmscited by 8
  33. 2019A Benchmark for Edge-Preserving Image Smoothingcited by 6
  34. 2019Least Squares Auto-Tuningcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 5; 2025: 1.

8 papers.

  1. 2024OGNI-DC: Robust Depth Completion with Optimization-Guided Neural Iterationscited by 23
  2. 2024PDiscoFormer: Relaxing Part Discovery Constraints with Vision Transformerscited by 9
  3. 2025Cross-Frame Representation Alignment for Fine-Tuning Video Diffusion Modelscited by 9
  4. 2024QCQP-Net: Reliably Learning Feasible Alternating Current Optimal Power Flow Solutions Under Constraintscited by 5
  5. 2023DFWLayer: Differentiable Frank-Wolfe Optimization Layercited by 2
  6. 2024Physically Based Neural Bidirectional Reflectance Distribution Functioncited by 1
  7. 2023A Novel Sparse Regularizercited by 0
  8. 2024Robust Network Learning via Inverse Scale Variational Sparsificationcited by 0