# VICTR: Visual Information Captured Text Representation for Text-to-Image Multimodal Tasks

arXiv: https://arxiv.org/abs/2010.03182 · HTML: https://mlsanity.com/paper/2010.03182/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 12 of 12)

- [Local Interpretations for Explainable Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.11072/) (2021)
- [MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering](https://mlsanity.com/paper/2503.18491/) (2025)
- [Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis](https://mlsanity.com/paper/2208.10970/) (2022)
- [Multimodal Commonsense Knowledge Distillation for Visual Question Answering](https://mlsanity.com/paper/2411.02722/) (2024)
- [Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation](https://mlsanity.com/paper/2209.14046/) (2022)
- [SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling](https://mlsanity.com/paper/2402.00319/) (2024)
- [Understanding Attention for Vision-and-Language Tasks](https://mlsanity.com/paper/2208.08104/) (2022)
- [PiggyBack: Pretrained Visual Question Answering Environment for Backing up Non-deep Learning Professionals](https://mlsanity.com/paper/2211.15940/) (2022)
- [SceneGATE: Scene-Graph based co-Attention networks for TExt visual question answering](https://mlsanity.com/paper/2212.08283/) (2022)
- [MSG-Chart: Multimodal Scene Graph for ChartQA](https://mlsanity.com/paper/2408.04852/) (2024)
- [3M-Health: Multimodal Multi-Teacher Knowledge Distillation for Mental Health Detection](https://mlsanity.com/paper/2407.09020/) (2024)
- [From Image Captioning to Visual Storytelling](https://mlsanity.com/paper/2508.14045/) (2025)

## Abstract (arXiv)

Text-to-image multimodal tasks, generating/retrieving an image from a given text description, are extremely challenging tasks since raw text descriptions cover quite limited information in order to fully describe visually realistic images. We propose a new visual contextual text representation for text-to-image multimodal tasks, VICTR, which captures rich visual semantic information of objects from the text input. First, we use the text description as initial input and conduct dependency parsing to extract the syntactic structure and analyse the semantic aspect, including object quantities, to extract the scene graph. Then, we train the extracted objects, attributes, and relations in the scene graph and the corresponding geometric relation information using Graph Convolutional Networks, and it generates text representation which integrates textual and visual semantic information. The text representation is aggregated with word-level and sentence-level embedding to generate both visual contextual word and sentence representation. For the evaluation, we attached VICTR to the state-of-the-art models in text-to-image generation.VICTR is easily added to existing models and improves across both quantitative and qualitative aspects.