# Not Just What's There: Enabling CLIP to Comprehend Negated Visual Descriptions Without Fine-tuning

arXiv: https://arxiv.org/abs/2602.21035 · HTML: https://mlsanity.com/paper/2602.21035/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [CREPE: Can Vision-Language Foundation Models Reason Compositionally?](https://mlsanity.com/paper/2212.07796/) (2022)
- [Alpha-CLIP: A CLIP Model Focusing on Wherever You Want](https://mlsanity.com/paper/2312.03818/) (2023)
- [Revisiting Catastrophic Forgetting in Large Language Model Tuning](https://mlsanity.com/paper/2406.04836/) (2024)
- [Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Data](https://mlsanity.com/paper/2105.07536/) (2021)
- [StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentation](https://mlsanity.com/paper/2408.01343/) (2024)
- [RpBERT: A Text-image Relation Propagation-based BERT Model for Multimodal NER](https://mlsanity.com/paper/2102.02967/) (2021)
- [SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasis](https://mlsanity.com/paper/2412.16563/) (2024)
- [CLIP Behaves like a Bag-of-Words Model Cross-modally but not Uni-modally](https://mlsanity.com/paper/2502.03566/) (2025)
- [Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing](https://mlsanity.com/paper/2509.12040/) (2025)
- [Know "No" Better: A Data-Driven Approach for Enhancing Negation Awareness in CLIP](https://mlsanity.com/paper/2501.10913/) (2025)
- [TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training](https://mlsanity.com/paper/2312.12828/) (2023)
- [EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation](https://mlsanity.com/paper/2504.09209/) (2025)
- [How and where does CLIP process negation?](https://mlsanity.com/paper/2407.10488/) (2024)
- [Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrieval](https://mlsanity.com/paper/2503.01980/) (2025)
- [MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference](https://mlsanity.com/paper/2506.15724/) (2025)
- [CLIPVQA:Video Quality Assessment via CLIP](https://mlsanity.com/paper/2407.04928/) (2024)
- [Bringing CLIP to the Clinic: Dynamic Soft Labels and Negation-Aware Learning for Medical Analysis](https://mlsanity.com/paper/2505.22079/) (2025)
- [Extract Free Dense Misalignment from CLIP](https://mlsanity.com/paper/2412.18404/) (2024)
- [FedCFA: Alleviating Simpson's Paradox in Model Aggregation with Counterfactual Federated Learning](https://mlsanity.com/paper/2412.18904/) (2024)
- [SPJFNet: Self-Mining Prior-Guided Joint Frequency Enhancement for Ultra-Efficient Dark Image Restoration](https://mlsanity.com/paper/2508.04041/) (2025)
- [Beyond Illumination: Fine-Grained Detail Preservation in Extreme Dark Image Restoration](https://mlsanity.com/paper/2508.03336/) (2025)

## Built on by (most cited first, 5 of 5)

- [CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges](https://mlsanity.com/paper/2603.11863/) (2026)
- [NEGATE: Constrained Semantic Guidance for Linguistic Negation in Text-to-Video Diffusion](https://mlsanity.com/paper/2603.06533/) (2026)
- [DCR: Counterfactual Attractor Guidance for Rare Compositional Generation](https://mlsanity.com/paper/2605.06512/) (2026)
- [TopoAgent: An Agentic Framework for Automated Topology Learning in Medical Imaging](https://mlsanity.com/paper/2606.29763/) (2026)
- [Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering](https://mlsanity.com/paper/2608.21450/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) like CLIP struggle to understand negation, often embedding affirmatives and negatives similarly (e.g., matching "no dog" with dog images). Existing methods refine negation understanding via fine-tuning CLIP's text encoder, risking overfitting. In this work, we propose CLIPGlasses, a plug-and-play framework that enhances CLIP's ability to comprehend negated visual descriptions. CLIPGlasses adopts a dual-stage design: a Lens module disentangles negated semantics from text embeddings, and a Frame module predicts context-aware repulsion strength, which is integrated into a modified similarity computation to penalize alignment with negated semantics, thereby reducing false positive matches. Experiments show that CLIP equipped with CLIPGlasses achieves competitive in-domain performance and outperforms state-of-the-art methods in cross-domain generalization. Its superiority is especially evident under low-resource conditions, indicating stronger robustness across domains.