Generated Contents Enrichment
Not on the map. Why?
Abstract (arXiv)
We study Generated Contents Enrichment (GCE), a conditional image-generation task in which a sparse scene description is first enriched through an explicit scene representation and then rendered into semantically richer visual content. Conventional image-generation systems can produce visually realistic outputs from limited scene descriptions, but the added content is usually implicit in the generator rather than represented as an inspectable intermediate structure. In contrast, GCE seeks to make scene enrichment explicit at the scene-representation level while examining its visual consequences during generation, with the goal of encouraging generated content that is visually plausible, structurally coherent, and semantically richer than the sparse input. To instantiate GCE, we propose a jointly trained adversarial framework that enriches scene graphs by modeling object semantics and inter-object relations. Our approach first represents the input description as a scene graph, where nodes model objects and edges capture inter-object relations. The framework uses graph convolutional networks to predict additional objects and their relations to the existing scene. Finally, the enriched scene graph is passed through the downstream image-generation pipeline to generate the corresponding visual content. We evaluate the framework with proxy scene graph enrichment metrics, image-quality comparisons, qualitative examples, and user studies on the Visual Genome dataset.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 107 of its 140 references that are in the map. First posted on arXiv 6 May 2024.
Builds on
107 of 140 references.
References · 107
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023GPT-4 Technical Reportcited by 18,132
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
38 more
- 2017Graph Attention Networkscited by 8,854
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Generative Adversarial Networkscited by 5,603
- 2016Layer Normalizationcited by 5,579
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2016Improved Techniques for Training GANscited by 2,938
- 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
- 2014Conditional Generative Adversarial Netscited by 2,152
- 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
- 2019Generating Long Sequences with Sparse Transformerscited by 1,523
- 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
- 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
- 2023PaLM 2 Technical Reportcited by 1,154
- 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
- 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
- 2016Conditional Image Generation with PixelCNN Decoderscited by 859
- 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
- 2016Pixel Recurrent Neural Networkscited by 814
- 2015Gated Graph Sequence Neural Networkscited by 792
- 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
- 2016Generative Adversarial Text to Image Synthesiscited by 735
- 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
- 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
- 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
50 most cited of 107 All 107 (JSON)
Built on by
Not cited yet.