# Consistency of Compositional Generalization across Multiple Levels

arXiv: https://arxiv.org/abs/2412.13636 · HTML: https://mlsanity.com/paper/2412.13636/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [Generative Multimodal Models are In-Context Learners](https://mlsanity.com/paper/2312.13286/) (2023)
- [Unifying Vision-and-Language Tasks via Text Generation](https://mlsanity.com/paper/2102.02779/) (2021)
- [Optimizing Millions of Hyperparameters by Implicit Differentiation](https://mlsanity.com/paper/1911.02590/) (2019)
- [Compositional Attention Networks for Machine Reasoning](https://mlsanity.com/paper/1803.03067/) (2018)
- [Multilingual Constituency Parsing with Self-Attention and Pre-Training](https://mlsanity.com/paper/1812.11760/) (2018)
- [Cycle-Consistency for Robust Visual Question Answering](https://mlsanity.com/paper/1902.05660/) (2019)
- [Compositional Temporal Grounding with Structured Variational Cross-Graph Correspondence Learning](https://mlsanity.com/paper/2203.13049/) (2022)
- [Language-Conditioned Graph Networks for Relational Reasoning](https://mlsanity.com/paper/1905.04405/) (2019)
- [Compositional Generalization by Learning Analytical Expressions](https://mlsanity.com/paper/2006.10627/) (2020)
- [Meta Module Network for Compositional Visual Reasoning](https://mlsanity.com/paper/1910.03230/) (2019)
- [Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation](https://mlsanity.com/paper/1909.04696/) (2019)
- [Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language](https://mlsanity.com/paper/2012.02646/) (2020)
- [Betty: An Automatic Differentiation Library for Multilevel Optimization](https://mlsanity.com/paper/2207.02849/) (2022)
- [Learning Compositional Neural Programs with Recursive Tree Search and Planning](https://mlsanity.com/paper/1905.12941/) (2019)
- [Evaluating Quality of Chatbots and Intelligent Conversational Agents](https://mlsanity.com/paper/1704.04579/) (2017)
- [ViewFusion: Towards Multi-View Consistency via Interpolated Denoising](https://mlsanity.com/paper/2402.18842/) (2024)
- [Logical Implications for Visual Question Answering Consistency](https://mlsanity.com/paper/2303.09427/) (2023)

## Built on by (most cited first, 1 of 1)

- [Differentiate-and-Inject: Enhancing VLAs via Functional Differentiation Induced by In-Parameter Structural Reasoning](https://mlsanity.com/paper/2602.07541/) (2026)

## Abstract (arXiv)

Compositional generalization is the capability of a model to understand novel compositions composed of seen concepts. There are multiple levels of novel compositions including phrase-phrase level, phrase-word level, and word-word level. Existing methods achieve promising compositional generalization, but the consistency of compositional generalization across multiple levels of novel compositions remains unexplored. The consistency refers to that a model should generalize to a phrase-phrase level novel composition, and phrase-word/word-word level novel compositions that can be derived from it simultaneously. In this paper, we propose a meta-learning based framework, for achieving consistent compositional generalization across multiple levels. The basic idea is to progressively learn compositions from simple to complex for consistency. Specifically, we divide the original training set into multiple validation sets based on compositional complexity, and introduce multiple meta-weight-nets to generate sample weights for samples in different validation sets. To fit the validation sets in order of increasing compositional complexity, we optimize the parameters of each meta-weight-net independently and sequentially in a multilevel optimization manner. We build a GQA-CCG dataset to quantitatively evaluate the consistency. Experimental results on visual question answering and temporal video grounding, demonstrate the effectiveness of the proposed framework. We release GQA-CCG at https://github.com/NeverMoreLCH/CCG.