mlsanityData to 7 Oct 2026

Centered Masking for Language-Image Pre-Training

arXiv 2403.15837 ↗First posted 23 Mar 2024cs.CV cs.CL cs.LGcited by 4

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

We introduce Gaussian masking for Language-Image Pre-Training (GLIP) a novel, straightforward, and effective technique for masking image patches during pre-training of a vision-language model. GLIP builds on Fast Language-Image Pre-Training (FLIP), which randomly masks image patches while training a CLIP model. GLIP replaces random masking with centered masking, that uses a Gaussian distribution and is inspired by the importance of image patches at the center of the image. GLIP retains the same computational savings as FLIP, while improving performance across a range of downstream datasets and tasks, as demonstrated by our experimental results. We show the benefits of GLIP to be easy to obtain, requiring no delicate tuning of the Gaussian, and also applicable to data sets containing images without an obvious center focus.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 38 references that are in the map. First posted on arXiv 23 Mar 2024.

Builds on

25 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
12 more
  1. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  2. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  3. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  4. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  5. 2015Attention-Based Models for Speech Recognitioncited by 400
  6. 2014Recurrent Models of Visual Attentioncited by 344
  7. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  8. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  9. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  10. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  11. 2017Dataset Augmentation in Feature Spacecited by 124
  12. 2022Attentive Mask CLIPcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Seeing What Matters: Empowering CLIP with Patch Generation-to-Selectioncited by 7
  2. 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replaycited by 4
  3. 2025Myna: Masking-Based Contrastive Learning of Musical Representationscited by 3
  4. 2026Dynamic Cluster Data Sampling for Efficient and Long-Tail-Aware Vision-Language Pre-trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Vision-Language Pre-training by Cluster Masking
  2. 2026GoldiCLIP: The Goldilocks Approach for Balancing Explicit Supervision for Language-Image Pretraining
  3. 2023ELIP: Efficient Discriminative Language-Image Pre-training with Fewer Vision Tokens
  4. 2022Uniform Masking Prevails in Vision-Language Pretraining
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
  6. 2022Leveraging per Image-Token Consistency for Vision-Language Pre-training
  7. 2021Data Efficient Masked Language Modeling for Vision and Language
  8. 2025MaskedCLIP: Bridging the Masked and CLIP Space for Semi-Supervised Medical Vision-Language Pre-training