mlsanityData to 7 Oct 2026

Generated Contents Enrichment

arXiv 2405.03650 ↗First posted 6 May 2024cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study Generated Contents Enrichment (GCE), a conditional image-generation task in which a sparse scene description is first enriched through an explicit scene representation and then rendered into semantically richer visual content. Conventional image-generation systems can produce visually realistic outputs from limited scene descriptions, but the added content is usually implicit in the generator rather than represented as an inspectable intermediate structure. In contrast, GCE seeks to make scene enrichment explicit at the scene-representation level while examining its visual consequences during generation, with the goal of encouraging generated content that is visually plausible, structurally coherent, and semantically richer than the sparse input. To instantiate GCE, we propose a jointly trained adversarial framework that enriches scene graphs by modeling object semantics and inter-object relations. Our approach first represents the input description as a scene graph, where nodes model objects and edges capture inter-object relations. The framework uses graph convolutional networks to predict additional objects and their relations to the existing scene. Finally, the enriched scene graph is passed through the downstream image-generation pipeline to generate the corresponding visual content. We evaluate the framework with proxy scene graph enrichment metrics, image-quality comparisons, qualitative examples, and user studies on the Visual Genome dataset.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 107 of its 140 references that are in the map. First posted on arXiv 6 May 2024.

Builds on

107 of 140 references.

References · 107

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2023GPT-4 Technical Reportcited by 18,132
  8. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  9. 2022Training language models to follow instructions with human feedbackcited by 15,712
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  11. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
38 more
  1. 2017Graph Attention Networkscited by 8,854
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2016Layer Normalizationcited by 5,579
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  13. 2016Improved Techniques for Training GANscited by 2,938
  14. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  15. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  16. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  17. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  18. 2014Conditional Generative Adversarial Netscited by 2,152
  19. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  20. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  21. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  22. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  23. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  24. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  25. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  26. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  27. 2023PaLM 2 Technical Reportcited by 1,154
  28. 2018Graph Convolutional Neural Networks for Web-Scale Recommender Systemscited by 1,133
  29. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  30. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  31. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  32. 2016Pixel Recurrent Neural Networkscited by 814
  33. 2015Gated Graph Sequence Neural Networkscited by 792
  34. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 751
  35. 2016Generative Adversarial Text to Image Synthesiscited by 735
  36. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  37. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  38. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651

50 most cited of 107 All 107 (JSON)

Built on by

Not cited yet.