mlsanityData to 7 Oct 2026

Semantic Image Synthesis via Adversarial Learning

arXiv 1707.06873 ↗First posted 21 Jul 2017cs.CVcited by 87

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a way of synthesizing realistic images directly with natural language description, which has many useful applications, e.g. intelligent image manipulation. We attempt to accomplish such synthesis: given a source image and a target text description, our model synthesizes images to meet two requirements: 1) being realistic while matching the target text description; 2) maintaining other image features that are irrelevant to the text description. The model should be able to disentangle the semantic information from the two modalities (image and text), and generate new images from the combined semantics. To achieve this, we proposed an end-to-end neural architecture that leverages adversarial learning to automatically learn implicit loss functions, which are optimized to fulfill the aforementioned two requirements. We have evaluated our model by conducting experiments on Caltech-200 bird dataset and Oxford-102 flower dataset, and have demonstrated that our model is capable of synthesizing realistic images that match the given descriptions, while still maintain other features of original images.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 20; 2022: 23; 2023: 28; 2024: 12; 2025: 4.

87 papers.

Sort:

By area:

  1. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  2. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  3. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  4. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  5. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
  6. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  7. 2023Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generationcited by 60
  8. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  9. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  10. 2021StyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulationcited by 40
  11. 2023Highly Personalized Text Embedding for Image Manipulation by Stable Diffusioncited by 39
  12. 2023DreamEdit: Subject-driven Image Editingcited by 37
38 more
  1. 2021M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformerscited by 36
  2. 2021Sound-Guided Semantic Image Manipulationcited by 35
  3. 2021Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Modelcited by 34
  4. 2022AnyFace: Free-style Text-to-Face Synthesis and Manipulationcited by 33
  5. 2023High-Fidelity 3D Face Generation from Natural Language Descriptionscited by 32
  6. 2021Cycle-Consistent Inverse GAN for Text-to-Image Synthesiscited by 27
  7. 2023DeltaEdit: Exploring Text-free Training for Text-Driven Image Manipulationcited by 27
  8. 2021Towards Open-World Text-Guided Face Image Generation and Manipulationcited by 25
  9. 2021Learning by Planning: Language-Guided Global Image Editingcited by 25
  10. 2023Region-Aware Diffusion for Zero-shot Text-driven Image Editingcited by 25
  11. 2023Improving Compositional Text-to-image Generation with Large Vision-Language Modelscited by 24
  12. 2022Learning Visual Styles from Audio-Visual Associationscited by 22
  13. 2022Word-Level Fine-Grained Story Visualizationcited by 22
  14. 2023FICE: Text-Conditioned Fashion Image Editing With Guided GAN Inversioncited by 21
  15. 2021Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Visioncited by 18
  16. 2023Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generationcited by 15
  17. 2021CLEVR_HYP: A Challenge Dataset and Baselines for Visual Question Answering with Hypothetical Actions over Imagescited by 14
  18. 2021PanoDR: Spherical Panorama Diminished Reality for Indoor Scenescited by 13
  19. 2022ManiTrans: Entity-Level Text-Guided Image Manipulation via Token-wise Semantic Alignment and Generationcited by 13
  20. 2022DE-Net: Dynamic Text-guided Image Editing Adversarial Networkscited by 12
  21. 2023Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Imagescited by 12
  22. 2025A Diffusion Model Translator for Efficient Image-to-Image Translationcited by 12
  23. 2021AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Stylecited by 11
  24. 2022Memory-Driven Text-to-Image Generationcited by 10
  25. 2022CLIP2GAN: Towards Bridging Text with the Latent Space of GANscited by 8
  26. 20223D-TOGO: Towards Text-Guided Cross-Category 3D Object Generationcited by 8
  27. 2024Self-Supervised Audio-Visual Soundscape Stylizationcited by 8
  28. 2021StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesiscited by 7
  29. 2022Towards Intrinsic Common Discriminative Features Learning for Face Forgery Detection using Adversarial Learningcited by 7
  30. 2022Robust Sound-Guided Image Manipulationcited by 7
  31. 2022AI Illustrator: Translating Raw Descriptions into Images by Prompt-based Cross-Modal Generationcited by 7
  32. 2022Modular Degradation Simulation and Restoration for Under-Display Cameracited by 7
  33. 2023Learning Profitable NFT Image Diffusions via Multiple Visual-Policy Guided Reinforcement Learningcited by 7
  34. 2023Language-based Photo Color Adjustment for Graphic Designscited by 6
  35. 2024Repositioning the Subject within Imagecited by 6
  36. 2024Implicit Multi-Spectral Transformer: An Lightweight and Effective Visible to Infrared Image Translation Modelcited by 6
  37. 2022Interactive Image Manipulation with Complex Text Instructionscited by 5
  38. 2023CoralStyleCLIP: Co-optimized Region and Layer Selection for Image Editingcited by 5

50 most cited of 87 All 87 (JSON)