# RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Network

arXiv: https://arxiv.org/abs/2104.03015 · HTML: https://mlsanity.com/paper/2104.03015/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 7 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding](https://mlsanity.com/paper/1606.01847/) (2016)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [MUTAN: Multimodal Tucker Fusion for Visual Question Answering](https://mlsanity.com/paper/1705.06676/) (2017)
- [Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering](https://mlsanity.com/paper/1708.01471/) (2017)
- [Dialog-based Interactive Image Retrieval](https://mlsanity.com/paper/1805.00145/) (2018)
- [Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering](https://mlsanity.com/paper/1708.03619/) (2017)
- [BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detection](https://mlsanity.com/paper/1902.00038/) (2019)
- [Compositional Learning of Image-Text Query for Image Retrieval](https://mlsanity.com/paper/2006.11149/) (2020)
- [Modality-Agnostic Attention Fusion for visual search with text feedback](https://mlsanity.com/paper/2007.00145/) (2020)
- [L2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks](https://mlsanity.com/paper/2003.13606/) (2020)
- [Multimodal Residual Learning for Visual QA](https://mlsanity.com/paper/1606.01455/) (2016)
- [Neural Naturalist: Generating Fine-Grained Image Comparisons](https://mlsanity.com/paper/1909.04101/) (2019)
- [Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction](https://mlsanity.com/paper/1511.05756/) (2015)
- [CurlingNet: Compositional Learning between Images and Text for Fashion IQ Data](https://mlsanity.com/paper/2003.12299/) (2020)
- [Semi-supervised Feature-Level Attribute Manipulation for Fashion Image Retrieval](https://mlsanity.com/paper/1907.05007/) (2019)
- [OD-GCN: Object Detection Boosted by Knowledge GCN](https://mlsanity.com/paper/1908.04385/) (2019)
- [Fashion-IQ 2020 Challenge 2nd Place Team's Solution](https://mlsanity.com/paper/2007.06404/) (2020)

## Built on by (most cited first, 27 of 27)

- [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023)
- [CompoDiff: Versatile Composed Image Retrieval With Latent Diffusion](https://mlsanity.com/paper/2303.11916/) (2023)
- [Sentence-level Prompts Benefit Composed Image Retrieval](https://mlsanity.com/paper/2310.05473/) (2023)
- [Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features](https://mlsanity.com/paper/2308.11485/) (2023)
- [CoVR-2: Automatic Data Construction for Composed Video Retrieval](https://mlsanity.com/paper/2308.14746/) (2023)
- [Data Roaming and Quality Assessment for Composed Image Retrieval](https://mlsanity.com/paper/2303.09429/) (2023)
- [Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization](https://mlsanity.com/paper/2211.07394/) (2022)
- [FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks](https://mlsanity.com/paper/2303.02483/) (2023)
- [FashionViL: Fashion-Focused Vision-and-Language Representation Learning](https://mlsanity.com/paper/2207.08150/) (2022)
- [Language-only Efficient Training of Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2312.01998/) (2023)
- [Composed Video Retrieval via Enriched Context and Discriminative Embeddings](https://mlsanity.com/paper/2403.16997/) (2024)
- [iSEARLE: Improving Textual Inversion for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2405.02951/) (2024)
- [CaLa: Complementary Association Learning for Augmenting Composed Image Retrieval](https://mlsanity.com/paper/2405.19149/) (2024)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [Dynamic Weighted Combiner for Mixed-Modal Image Retrieval](https://mlsanity.com/paper/2312.06179/) (2023)
- [Large-scale Bilingual Language-Image Contrastive Learning](https://mlsanity.com/paper/2203.14463/) (2022)
- [Image Search with Text Feedback by Additive Attention Compositional Learning](https://mlsanity.com/paper/2203.03809/) (2022)
- [UIGR: Unified Interactive Garment Retrieval](https://mlsanity.com/paper/2204.03111/) (2022)
- [Language Guided Local Infiltration for Interactive Image Retrieval](https://mlsanity.com/paper/2304.07747/) (2023)
- [Beyond Simple Edits: Composed Video Retrieval with Dense Modifications](https://mlsanity.com/paper/2508.14039/) (2025)
- [Training and challenging models for text-guided fashion image retrieval](https://mlsanity.com/paper/2204.11004/) (2022)
- [Dual Relation Alignment for Composed Image Retrieval](https://mlsanity.com/paper/2309.02169/) (2023)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrieval](https://mlsanity.com/paper/2207.04211/) (2022)
- [TMCIR: Token Merge Benefits Composed Image Retrieval](https://mlsanity.com/paper/2504.10995/) (2025)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [Agentic Personalized Fashion Recommendation in the Age of Generative AI: Challenges, Opportunities, and Evaluation](https://mlsanity.com/paper/2508.02342/) (2025)

## Abstract (arXiv)

In this paper, we study the compositional learning of images and texts for image retrieval. The query is given in the form of an image and text that describes the desired modifications to the image; the goal is to retrieve the target image that satisfies the given modifications and resembles the query by composing information in both the text and image modalities. To remedy this, we propose a novel architecture designed for the image-text composition task and show that the proposed structure can effectively encode the differences between the source and target images conditioned on the text. Furthermore, we introduce a new joint training technique based on the graph convolutional network that is generally applicable for any existing composition methods in a plug-and-play manner. We found that the proposed technique consistently improves performance and achieves state-of-the-art scores on various benchmarks. To avoid misleading experimental results caused by trivial training hyper-parameters, we reproduce all individual baselines and train models with a unified training environment. We expect this approach to suppress undesirable effects from irrelevant components and emphasize the image-text composition module's ability. Also, we achieve the state-of-the-art score without restricting the training environment, which implies the superiority of our method considering the gains from hyper-parameter tuning. The code, including all the baseline methods, are released https://github.com/nashory/rtic-gcn-pytorch.