mlsanityData to 7 Oct 2026

Unsupervised Foreground Extraction via Deep Region Competition

arXiv 2110.15497 ↗First posted 29 Oct 2021cs.CV cs.LGcited by 26

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

We present Deep Region Competition (DRC), an algorithm designed to extract foreground objects from images in a fully unsupervised manner. Foreground extraction can be viewed as a special case of generic image segmentation that focuses on identifying and disentangling objects from the background. In this work, we rethink the foreground extraction by reconciling energy-based prior with generative image modeling in the form of Mixture of Experts (MoE), where we further introduce the learned pixel re-assignment as the essential inductive bias to capture the regularities of background regions. With this modeling, the foreground-background partition can be naturally found through Expectation-Maximization (EM). We show that the proposed method effectively exploits the interaction between the mixture components during the partitioning process, which closely connects to region competition, a seminal approach for generic image segmentation. Experiments demonstrate that DRC exhibits more competitive performances on complex real-world data and challenging multi-object scenes compared with prior methods. Moreover, we show empirically that DRC can potentially generalize to novel foreground objects even from categories unseen during training.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 52 of its 102 references that are in the map. First posted on arXiv 29 Oct 2021.

Builds on

52 of 102 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2017Mask R-CNNcited by 7,259
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  9. 2013Auto-Encoding Variational Bayescited by 3,980
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  11. 2016Pyramid Scene Parsing Networkcited by 2,472
  12. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
38 more
  1. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  3. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  4. 2017Deep Image Priorcited by 1,097
  5. 2020Object-Centric Learning with Slot Attentioncited by 865
  6. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  7. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  8. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  9. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  10. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  11. 2015BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentationcited by 238
  12. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  13. 2016Simple Does It: Weakly Supervised Instance and Semantic Segmentationcited by 226
  14. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  15. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  16. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  17. 2015Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentationcited by 189
  18. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
  19. 2017Neural Expectation Maximizationcited by 123
  20. 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
  21. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  22. 2015Constrained Convolutional Neural Networks for Weakly Supervised Segmentationcited by 100
  23. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  24. 2018Iterative Amortized Inferencecited by 93
  25. 2014Salient Object Detection: A Discriminative Regional Feature Integration Approachcited by 89
  26. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  27. 2019Unsupervised Moving Object Detection via Contextual Information Separationcited by 72
  28. 2016Tagger: Deep Unsupervised Perceptual Groupingcited by 69
  29. 2019Unsupervised Part-Based Disentangling of Object Shape and Appearancecited by 67
  30. 2018FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discoverycited by 58
  31. 2017W-Net: A Deep Model for Fully Unsupervised Image Segmentationcited by 57
  32. 2019Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentationcited by 57
  33. 2020Object Segmentation Without Labels with Large-Scale Generative Modelscited by 52
  34. 2017LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generationcited by 36
  35. 2017Exploiting saliency for object segmentation from image level labelscited by 33
  36. 2019OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clusteringcited by 28
  37. 2020Information-Theoretic Segmentation by Inpainting Error Maximizationcited by 22
  38. 2021Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classificationcited by 14

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 10; 2023: 9; 2024: 7.

26 papers.

Sort:

By area:

  1. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  2. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
  3. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  4. 2024Flow Priors for Linear Inverse Problems via Iterative Corrupted Trajectory Matchingcited by 50
  5. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  6. 2022Discovering Objects that Can Movecited by 44
  7. 2022Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motioncited by 42
  8. 2022Neural Systematic Bindercited by 37
  9. 2022EM-driven unsupervised learning for efficient motion segmentationcited by 33
  10. 2023Object Discovery from Motion-Guided Tokenscited by 25
  11. 2022Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Modelcited by 18
  12. 2024Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Modelscited by 18
14 more
  1. 2022RelViT: Concept-guided Vision Transformer for Visual Relational Reasoningcited by 14
  2. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  3. 2023Weakly-supervised Contrastive Learning for Unsupervised Object Discoverycited by 12
  4. 2024"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihoodcited by 8
  5. 2023Unsupervised Skin Lesion Segmentation via Structural Entropy Minimization on Multi-Scale Superpixel Graphscited by 7
  6. 2024Watermarking Generative Tabular Datacited by 6
  7. 2023SC-VAE: Sparse Coding-based Variational Autoencoder with Learned ISTAcited by 4
  8. 2023Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentationcited by 4
  9. 2024Latent Space Energy-based Neural ODEscited by 4
  10. 2024Unsupervised Object Discovery: A Comprehensive Survey and Unified Taxonomycited by 4
  11. 2023CMVAE: Causal Meta VAE for Unsupervised Meta-Learningcited by 3
  12. 2024Dexterous Functional Pre-Grasp Manipulation with Diffusion Policycited by 3
  13. 2022ILSGAN: Independent Layer Synthesis for Unsupervised Foreground-Background Segmentationcited by 2
  14. 2023USL-Net: Uncertainty Self-Learning Network for Unsupervised Skin Lesion Segmentationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2022Foreground-Background Separation through Concept Distillation from Generative Image Foundation Models
  2. 2024Unsupervised Segmentation by Diffusing, Walking and Cutting
  3. 2024A Simple Background Augmentation Method for Object Detection with Diffusion Model
  4. 2026Making Training-Free Diffusion Segmentors Scale with the Generative Power
  5. 2025OmnimatteZero: Fast Training-free Omnimatte with Pre-trained Video Diffusion Models
  6. 2023DDP: Diffusion Model for Dense Visual Prediction
  7. 2026Unified Video Dense Prediction from Disjoint Data
  8. 2025EraseLoRA: MLLM-Driven Foreground Exclusion and Background Subtype Aggregation for Dataset-Free Object Removal