# DisCoCLIP: A Distributional Compositional Tensor Network Encoder for Vision-Language Understanding

arXiv: https://arxiv.org/abs/2509.21287 · HTML: https://mlsanity.com/paper/2509.21287/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 53 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 16 of 16)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [Tensorizing Neural Networks](https://mlsanity.com/paper/1509.06569/) (2015)
- [Probing Image-Language Transformers for Verb Understanding](https://mlsanity.com/paper/2106.09141/) (2021)
- [Does CLIP Bind Concepts? Probing Compositionality in Large Image Models](https://mlsanity.com/paper/2212.10537/) (2022)
- [Logic Tensor Networks for Semantic Image Interpretation](https://mlsanity.com/paper/1705.08968/) (2017)
- [Learning Relevant Features of Data with Multi-scale Tensor Networks](https://mlsanity.com/paper/1801.00315/) (2017)
- [lambeq: An Efficient High-Level Python Library for Quantum NLP](https://mlsanity.com/paper/2110.04236/) (2021)
- [ComCLIP: Training-Free Compositional Image and Text Matching](https://mlsanity.com/paper/2211.13854/) (2022)
- [Tensor Networks for Medical Image Classification](https://mlsanity.com/paper/2004.10076/) (2020)
- [A CCG-Based Version of the DisCoCat Framework](https://mlsanity.com/paper/2105.07720/) (2021)
- [Tensor Networks for Explainable Machine Learning in Cybersecurity](https://mlsanity.com/paper/2401.00867/) (2023)
- [A Generalized Language Model in Tensor Space](https://mlsanity.com/paper/1901.11167/) (2019)
- [Something Old, Something New: Grammar-based CCG Parsing with Transformer Models](https://mlsanity.com/paper/2109.10044/) (2021)

## Built on by (most cited first, 3 of 3)

- [Quantum Models with Multi-Stage Training for Compositional Concept Generalization](https://mlsanity.com/paper/2608.15601/) (2026)
- [The Limits of Binding in Dual Encoders](https://mlsanity.com/paper/2608.15971/) (2026)
- [Using Composition Operators to Linearize LLM Semantic Transformations](https://mlsanity.com/paper/2609.22143/) (2026)

## Abstract (arXiv)

Recent vision-language models excel at large-scale image-text alignment but often neglect the compositional structure of language, leading to failures on tasks that hinge on word order and predicate-argument structure. We introduce DisCoCLIP, a multimodal encoder that combines a frozen CLIP vision transformer with a novel tensor network text encoder that explicitly encodes syntactic structure. Sentences are parsed with a Combinatory Categorial Grammar parser to yield distributional word tensors whose contractions mirror the sentence's grammatical derivation. To keep the model efficient, high-order tensors are factorized with tensor decompositions, reducing parameter count from tens of millions to under one million. Trained end-to-end with a self-supervised contrastive loss, DisCoCLIP markedly improves sensitivity to verb semantics and word order: it raises CLIP's SVO-Probes verb accuracy from 77.6% to 82.4%, boosts ARO attribution and relation scores by over 9% and 4%, and achieves 93.7% on a newly introduced SVO-Swap benchmark. These results demonstrate that embedding explicit linguistic structure via tensor networks yields interpretable, parameter-efficient representations that substantially improve compositional reasoning in vision-language tasks.