# TNG-CLIP:Training-Time Negation Data Generation for Negation Awareness of CLIP

arXiv: https://arxiv.org/abs/2505.18434 · HTML: https://mlsanity.com/paper/2505.18434/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 24 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [Deep Learning is Robust to Massive Label Noise](https://mlsanity.com/paper/1705.10694/) (2017)
- [VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena](https://mlsanity.com/paper/2112.07566/) (2021)
- [Dialog-based Interactive Image Retrieval](https://mlsanity.com/paper/1805.00145/) (2018)
- [Language models are not naysayers: An analysis of language models on negation benchmarks](https://mlsanity.com/paper/2306.08189/) (2023)
- [Vision-Language Models Do Not Understand Negation](https://mlsanity.com/paper/2501.09425/) (2025)
- [Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws](https://mlsanity.com/paper/2410.11820/) (2024)
- [Learn "No" to Say "Yes" Better: Improving Vision-Language Models via Negations](https://mlsanity.com/paper/2403.20312/) (2024)
- [Know "No" Better: A Data-Driven Approach for Enhancing Negation Awareness in CLIP](https://mlsanity.com/paper/2501.10913/) (2025)
- [Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation](https://mlsanity.com/paper/2210.03256/) (2022)
- [How and where does CLIP process negation?](https://mlsanity.com/paper/2407.10488/) (2024)
- [Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negation](https://mlsanity.com/paper/2406.05494/) (2024)
- [Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models](https://mlsanity.com/paper/2305.17311/) (2023)
- [Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation](https://mlsanity.com/paper/2303.14423/) (2023)
- [WhittleSearch: Interactive Image Search with Relative Attribute Feedback](https://mlsanity.com/paper/1505.04141/) (2015)
- [Impact of Noisy Supervision in Foundation Model Learning](https://mlsanity.com/paper/2403.06869/) (2024)
- [Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation](https://mlsanity.com/paper/1805.00178/) (2018)
- [Modyn: Data-Centric Machine Learning Pipeline Orchestration](https://mlsanity.com/paper/2312.06254/) (2023)

## Built on by (most cited first, 7 of 7)

- [DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval](https://mlsanity.com/paper/2603.09185/) (2026)
- [NAST: Improving Negation Handling in Medical Vision-Language Models through Negation-Aware Selective Training](https://mlsanity.com/paper/2602.12498/) (2026)
- [Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?](https://mlsanity.com/paper/2603.00166/) (2026)
- [Benchmarking MLLMs via Cognitive Expected Scene Graph for Safety-Critical Visual Negation Understanding](https://mlsanity.com/paper/2609.19767/) (2026)
- [Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraint](https://mlsanity.com/paper/2609.19812/) (2026)
- [Skeleton-and-Strategy Prompting: Training-Free Negation Understanding for Vision-Language Models](https://mlsanity.com/paper/2610.01180/) (2026)
- [NegT2IBench: When Negation Changes the Picture. A Polarity Benchmark for Text-to-Image Models](https://mlsanity.com/paper/2610.03084/) (2026)

## Abstract (arXiv)

Vision-language models (VLMs), such as CLIP, have demonstrated strong performance across a range of downstream tasks. However, CLIP is still limited in negation understanding: the ability to recognize the absence or exclusion of a concept. Existing methods address the problem by using a large language model (LLM) to generate large-scale data of image captions containing negation for further fine-tuning CLIP. However, these methods are both time- and compute-intensive, and their evaluations are typically restricted to image-text matching tasks. To expand the horizon, we (1) introduce a training-time negation data generation pipeline such that negation captions are generated during the training stage, which only increases 2.5% extra training time, and (2) we propose the first benchmark, Neg-TtoI, for evaluating text-to-image generation models on prompts containing negation, assessing model's ability to produce semantically accurate images. We show that our proposed method, TNG-CLIP, achieves SOTA performance on diverse negation benchmarks of image-to-text matching, text-to-image retrieval, and image generation.