mlsanityData to 7 Oct 2026

Context Encoders: Feature Learning by Inpainting

arXiv 1604.07379 ↗First posted 25 Apr 2016cs.CV cs.AI cs.GRcited by 1,424

Not on the map. Why?

Cited by 76 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present an unsupervised visual feature learning algorithm driven by context-based pixel prediction. By analogy with auto-encoders, we propose Context Encoders -- a convolutional neural network trained to generate the contents of an arbitrary image region conditioned on its surroundings. In order to succeed at this task, context encoders need to both understand the content of the entire image, as well as produce a plausible hypothesis for the missing part(s). When training context encoders, we have experimented with both a standard pixel-wise reconstruction loss, as well as a reconstruction plus an adversarial loss. The latter produces much sharper results because it can better handle multiple modes in the output. We found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures. We quantitatively demonstrate the effectiveness of our learned features for CNN pre-training on classification, detection, and segmentation tasks. Furthermore, context encoders can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.

Key facts. Cited by 1,424 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 370; 2022: 373; 2023: 287; 2024: 191; 2025: 129; 2026: 72.

1,424 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  4. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  5. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  7. 2021Transformers in Vision: A Surveycited by 1,009
  8. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  9. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  10. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  11. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  12. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
38 more
  1. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  2. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  3. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  4. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  5. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  6. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  7. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 418
  8. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  9. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  10. 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
  11. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  12. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  13. 2021Graph Self-Supervised Learning: A Surveycited by 289
  14. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  15. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 287
  16. 2022Masked Autoencoders that Listencited by 271
  17. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 253
  18. 2022Visual Prompting via Image Inpaintingcited by 248
  19. 2023A Cookbook of Self-Supervised Learningcited by 248
  20. 2022Masked Visual Pre-training for Motor Controlcited by 246
  21. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  22. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  23. 2023TextDiffuser: Diffusion Models as Text Painterscited by 223
  24. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 222
  25. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
  26. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 216
  27. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  28. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  29. 2022Test-Time Training with Masked Autoencoderscited by 208
  30. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  31. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  32. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  33. 2023EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anythingcited by 187
  34. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  35. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  36. 2021BEVT: BERT Pretraining of Video Transformerscited by 169
  37. 2023A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpaintingcited by 168
  38. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 158

50 most cited of 1,424 All 1,424 (JSON)