# NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization

arXiv: https://arxiv.org/abs/2412.15588 · HTML: https://mlsanity.com/paper/2412.15588/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 50 references that are in the map.
- First posted on arXiv 20 Dec 2024.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models](https://mlsanity.com/paper/2308.01390/) (2023)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Binding Language Models in Symbolic Languages](https://mlsanity.com/paper/2210.02875/) (2022)
- [Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models](https://mlsanity.com/paper/2407.07035/) (2024)
- [Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions](https://mlsanity.com/paper/2212.10561/) (2022)
- [Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?](https://mlsanity.com/paper/2203.17271/) (2022)
- [Nonconvex Robust High-Order Tensor Completion Using Randomized Low-Rank Approximation](https://mlsanity.com/paper/2305.11495/) (2023)
- [Prompt2DeModel: Declarative Neuro-Symbolic Modeling with Natural Language](https://mlsanity.com/paper/2407.20513/) (2024)

## Built on by (most cited first, 13 of 13)

- [Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector](https://mlsanity.com/paper/2503.20188/) (2025)
- [NePTune: A Neuro-Pythonic Framework for Tunable Compositional Reasoning on Vision-Language](https://mlsanity.com/paper/2509.25757/) (2025)
- [On the Holistic Approach for Detecting Human Image Forgery](https://mlsanity.com/paper/2601.04715/) (2026)
- [Abductive Logical Rule Induction by Bridging Inductive Logic Programming and Multimodal Large Language Models](https://mlsanity.com/paper/2509.21874/) (2025)
- [Referring Expressions as a Lens into Spatial Language Grounding in Vision-Language Models](https://mlsanity.com/paper/2511.06146/) (2025)
- [An Agentic Framework for Neuro-Symbolic Programming](https://mlsanity.com/paper/2601.00743/) (2026)
- [An Alternative Trajectory for Generative AI](https://mlsanity.com/paper/2603.14147/) (2026)
- [Spatial Reasoning via Modality Switching Between Language and Symbolic Representations](https://mlsanity.com/paper/2606.31285/) (2026)
- [Logi-PAR: Logic-Infused Patient Activity Recognition via Differentiable Rule](https://mlsanity.com/paper/2603.05184/) (2026)
- [CRISP: Compositional Relations as Invariant Structural Priors for Domain Generalization](https://mlsanity.com/paper/2605.06043/) (2026)
- [Generating Workflow DAGs from Natural Language with Non-Reasoning LLMs](https://mlsanity.com/paper/2608.30250/) (2026)
- [CoGe-GCD: Reframing Generalized Category Discovery with Compositional Generalization](https://mlsanity.com/paper/2609.10158/) (2026)
- [Differentiable Fuzzy Inference Layer: A Monotone, Compositional Ordinal Reasoning Head for Large Language Models](https://mlsanity.com/paper/2609.26113/) (2026)

## Abstract (arXiv)

Compositional generalization is crucial for artificial intelligence agents to solve complex vision-language reasoning tasks. Neuro-symbolic approaches have demonstrated promise in capturing compositional structures, but they face critical challenges: (a) reliance on predefined predicates for symbolic representations that limit adaptability, (b) difficulty in extracting predicates from raw data, and (c) using non-differentiable operations for combining primitive concepts. To address these issues, we propose NeSyCoCo, a neuro-symbolic framework that leverages large language models (LLMs) to generate symbolic representations and map them to differentiable neural computations. NeSyCoCo introduces three innovations: (a) augmenting natural language inputs with dependency structures to enhance the alignment with symbolic representations, (b) employing distributed word representations to link diverse, linguistically motivated logical predicates to neural modules, and (c) using the soft composition of normalized predicate scores to align symbolic and differentiable reasoning. Our framework achieves state-of-the-art results on the ReaSCAN and CLEVR-CoGenT compositional generalization benchmarks and demonstrates robust performance with novel concepts in the CLEVR-SYN benchmark.