mlsanityData to 7 Oct 2026

ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing

arXiv 1803.01837 ↗First posted 5 Mar 2018cs.CV cs.LGcited by 90

Not on the map. Why?

Abstract (arXiv)

We address the problem of finding realistic geometric corrections to a foreground object such that it appears natural when composited into a background image. To achieve this, we propose a novel Generative Adversarial Network (GAN) architecture that utilizes Spatial Transformer Networks (STNs) as the generator, which we call Spatial Transformer GANs (ST-GANs). ST-GANs seek image realism by operating in the geometric warp parameter space. In particular, we exploit an iterative STN warping scheme and propose a sequential training strategy that achieves better results compared to naive training of a single generator. One of the key advantages of ST-GAN is its applicability to high-resolution images indirectly since the predicted warp parameters are transferable between reference frames. We demonstrate our approach in two applications: (1) visualizing how indoor furniture (e.g. from product images) might be perceived in a room, (2) hallucinating how accessories like glasses would look when matched with real portraits.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 18; 2023: 24; 2024: 14; 2025: 9; 2026: 3.

90 papers.

Sort:

By area:

  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  2. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  3. 2021Diverse Image Inpainting with Bidirectional and Autoregressive Transformerscited by 80
  4. 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
  5. 2024IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representationcited by 64
  6. 2021Making Images Real Again: A Comprehensive Survey on Deep Image Compositioncited by 63
  7. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  8. 2023Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generationcited by 60
  9. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  10. 2021Unsupervised Layered Image Decomposition into Object Prototypescited by 50
  11. 2023Turning a CLIP Model into a Scene Text Detectorcited by 44
  12. 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
38 more
  1. 2022ObjectStitch: Generative Object Compositingcited by 36
  2. 2021Shadow Generation for Composite Image in Real-world Scenescited by 35
  3. 2021MarioNette: Self-Supervised Sprite Learningcited by 35
  4. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  5. 2021Toward Accurate and Realistic Outfits Visualization with Attention to Detailscited by 33
  6. 2021Blind Image Super-Resolution via Contrastive Representation Learningcited by 32
  7. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 30
  8. 2022Probabilistic Compositional Embeddings for Multimodal Image Retrievalcited by 29
  9. 2021OPA: Object Placement Assessment Datasetcited by 28
  10. 2021Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognitioncited by 24
  11. 2021Compositional Transformers for Scene Generationcited by 24
  12. 2022Learning Object Placement via Dual-path Graph Completioncited by 20
  13. 2024PrimeComposer: Faster Progressively Combined Diffusion for Image Composition with Attention Steeringcited by 20
  14. 2024Thinking Outside the BBox: Unconstrained Generative Object Compositingcited by 19
  15. 2023A Data Perspective on Enhanced Identity Preservation for Diffusion Personalizationcited by 18
  16. 2023SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Controlcited by 18
  17. 2023NCHO: Unsupervised Learning for Neural 3D Composition of Humans and Objectscited by 17
  18. 2023DreamCom: Finetuning Text-guided Inpainting Model for Image Compositioncited by 17
  19. 2024Shadow Generation for Composite Image Using Diffusion modelcited by 17
  20. 2023TopNet: Transformer-based Object Placement Network for Image Compositingcited by 16
  21. 2023Integrating View Conditions for Image Synthesiscited by 16
  22. 2023Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generationcited by 15
  23. 20233D-aware Blending with Generative NeRFscited by 13
  24. 2023Cross-domain Compositing with Pretrained Diffusion Modelscited by 13
  25. 2021Deep Feedback Inverse Problem Solvercited by 11
  26. 2023VTAE: Variational Transformer Autoencoder with Manifolds Learningcited by 10
  27. 2023Scene-aware Human Pose Generation using Transformercited by 9
  28. 2023Turning a CLIP Model into a Scene Text Spottercited by 9
  29. 2021Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networkscited by 8
  30. 2021Repopulating Street Scenescited by 6
  31. 2021Unsupervised Image Decomposition with Phase-Correlation Networkscited by 6
  32. 2021GaussiGAN: Controllable Image Synthesis with 3D Gaussians from Unposed Silhouettescited by 5
  33. 2021SAC-GAN: Structure-Aware Image Compositioncited by 5
  34. 2022A Scene-Text Synthesis Engine Achieved Through Learning from Decomposed Real-World Datacited by 5
  35. 2023Controllable Image Generation via Collage Representationscited by 5
  36. 2022MICDIR: Multi-scale Inverse-consistent Deformable Image Registration using UNetMSS with Self-Constructing Graph Latentcited by 4
  37. 2022GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositingcited by 4
  38. 2022Vec2Face-v2: Unveil Human Faces from their Blackbox Features via Attention-based Network in Face Recognitioncited by 4

50 most cited of 90 All 90 (JSON)