mlsanityData to 7 Oct 2026

Regularized Frank-Wolfe for Dense CRFs: Generalizing Mean Field and Beyond

arXiv 2110.14759 ↗First posted 27 Oct 2021cs.LG cs.CV math.OCcited by 10

Not on the map. Why?

Abstract (arXiv)

We introduce regularized Frank-Wolfe, a general and effective algorithm for inference and learning of dense conditional random fields (CRFs). The algorithm optimizes a nonconvex continuous relaxation of the CRF inference problem using vanilla Frank-Wolfe with approximate updates, which are equivalent to minimizing a regularized energy function. Our proposed method is a generalization of existing algorithms such as mean field or concave-convex procedure. This perspective not only offers a unified analysis of these algorithms, but also allows an easy way of exploring different variants that potentially yield better performance. We illustrate this in our empirical results on standard semantic segmentation datasets, where several instantiations of our regularized Frank-Wolfe outperform mean field inference, both as a standalone component and as an end-to-end trainable layer in a neural network. We also show that dense CRFs, coupled with our new algorithms, produce significant improvements over strong CNN baselines.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 38 of its 92 references that are in the map. First posted on arXiv 27 Oct 2021.

Builds on

38 of 92 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  6. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  7. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  8. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  9. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 463
  10. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  11. 2019Differentiation of Blackbox Combinatorial Solverscited by 224
  12. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
26 more
  1. 2015Attention to Scale: Scale-aware Semantic Image Segmentationcited by 169
  2. 2014Simultaneous Detection and Segmentationcited by 158
  3. 2015On the Global Linear Convergence of Frank-Wolfe Optimization Variantscited by 76
  4. 2015Efficient piecewise training of deep structured models for semantic segmentationcited by 71
  5. 2017Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascadecited by 53
  6. 2015Semantic Image Segmentation via Deep Parsing Networkcited by 44
  7. 2017A Regularized Framework for Sparse and Structured Neural Attentioncited by 38
  8. 2013The Complexity of Large-scale Convex Programming under a Linear Optimization Oraclecited by 27
  9. 2016Fast, Exact and Multi-Scale Inference for Semantic Image Segmentation with Deep Gaussian CRFscited by 27
  10. 2015Higher Order Conditional Random Fields in Deep Neural Networkscited by 25
  11. 2015Fully Connected Deep Structured Networkscited by 24
  12. 2019The Limited Multi-Label Projection Layercited by 21
  13. 2013New Analysis and Results for the Frank-Wolfe Methodcited by 17
  14. 2014On learning to localize objects with minimal supervisioncited by 17
  15. 2013Conditional Gradient Algorithms for Norm-Regularized Smooth Convex Optimizationcited by 13
  16. 2017A Unified Optimization View on Generalized Matching Pursuit and Frank-Wolfecited by 12
  17. 2016Alternating Direction Graph Matchingcited by 9
  18. 2014Generalized Conditional Gradient for Sparse Estimationcited by 6
  19. 2015Principled Parallel Mean-Field Inference for Discrete Random Fieldscited by 6
  20. 2018Conditional Random Fields as Recurrent Neural Networks for 3D Medical Imaging Segmentationcited by 6
  21. 2018Proximal Mean-field for Neural Network Quantizationcited by 6
  22. 2020Affine Invariant Analysis of Frank-Wolfe on Strongly Convex Setscited by 6
  23. 2016Mixed context networks for semantic segmentationcited by 2
  24. 2016Efficient Continuous Relaxations for Dense CRFcited by 1
  25. 2016Efficient Linear Programming for Dense CRFscited by 1
  26. 2018Continuous Relaxation of MAP Inference: A Nonconvex Perspectivecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 0; 2025: 1; 2026: 3.

10 papers.

  1. 2023Loss Functions in the Era of Semantic Segmentation: A Survey and Outlookcited by 33
  2. 2026INSID3: Training-Free In-Context Segmentation with DINOv3cited by 12
  3. 2022Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Fieldcited by 4
  4. 2023Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representationcited by 4
  5. 2022Modeling Label Correlations for Second-Order Semantic Dependency Parsing with Mean-Field Inferencecited by 1
  6. 2022Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Fieldcited by 1
  7. 2025CRISP: A Framework for Cryo-EM Image Segmentation and Processing with Conditional Random Fieldcited by 1
  8. 2026Scene-Centric Unsupervised Video Panoptic Segmentationcited by 1
  9. 2023Monotone deep Boltzmann machinescited by 0
  10. 2026UnfoldCRF: Structured Mask Refinement with Image-Conditioned Latent Regionscited by 0