mlsanityData to 7 Oct 2026

Guided Co-Modulated GAN for 360deg Field of View Extrapolation

arXiv 2204.07286 ↗First posted 15 Apr 2022cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

We propose a method to extrapolate a 360deg field of view from a single image that allows for user-controlled synthesis of the out-painted content. To do so, we propose improvements to an existing GAN-based in-painting architecture for out-painting panoramic image representation. Our method obtains state-of-the-art results and outperforms previous methods on standard image quality metrics. To allow controlled synthesis of out-painting, we introduce a novel guided co-modulation framework, which drives the image generation process with a common pretrained discriminative model. Doing so maintains the high visual quality of generated panoramas while enabling user-controlled semantic content in the extrapolated field of view. We demonstrate the state-of-the-art results of our method on field of view extrapolation both qualitatively and quantitatively, providing thorough analysis of our novel editing capabilities. Finally, we demonstrate that our approach benefits the photorealistic virtual insertion of highly glossy objects in photographs.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 62 of its 80 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

62 of 80 references.

References · 62

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  10. 2014Conditional Generative Adversarial Netscited by 2,152
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  12. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
38 more
  1. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  2. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  3. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  4. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  5. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  6. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  7. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  8. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  9. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  10. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  11. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  12. 2021GAN Inversion: A Surveycited by 366
  13. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  14. 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 297
  15. 2020A Survey on Neural Network Interpretabilitycited by 291
  16. 2019On the "steerability" of generative adversarial networkscited by 247
  17. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  18. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
  19. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
  20. 2019GANalyze: Toward Visual Definitions of Cognitive Image Propertiescited by 193
  21. 2020Semantic Photo Manipulation with a Generative Image Priorcited by 178
  22. 2019Effectively Unbiased FID and Inception Score and where to find themcited by 161
  23. 2017Learning to Predict Indoor Illumination from a Single Imagecited by 160
  24. 2021Learning Indoor Inverse Rendering with 3D Spatially-Varying Lightingcited by 82
  25. 2018Patch-Based Image Inpainting with Generative Adversarial Networkscited by 70
  26. 2019COCO-GAN: Generation by Parts via Conditional Coordinatingcited by 67
  27. 2020Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illuminationcited by 62
  28. 2018DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applicationscited by 61
  29. 2021InfinityGAN: Towards Infinite-Pixel Image Synthesiscited by 61
  30. 2019Corners for Layout: End-to-End Layout Recovery from 360 Imagescited by 55
  31. 2021Using latent space regression to analyze and leverage compositionality in GANscited by 55
  32. 2021In&Out : Diverse Image Outpainting via GAN Inversioncited by 55
  33. 2018Semi-parametric Image Synthesiscited by 50
  34. 2019Very Long Natural Scenery Image Prediction by Outpaintingcited by 50
  35. 2019Learning Single Camera Depth Estimation using Dual-Pixelscited by 49
  36. 2022Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creationcited by 47
  37. 2020HDR Environment Map Estimation for Real-Time Augmented Realitycited by 46
  38. 2017A Perceptual Measure for Deep Single Image Camera Calibrationcited by 45

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 9; 2025: 12; 2026: 3.

32 papers.

Sort:

By area:

  1. 2023DiffusionLight: Light Probes for Free by Painting a Chrome Ballcited by 63
  2. 2024360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Modelcited by 56
  3. 2023360-Degree Panorama Generation from Few Unregistered NFoV Imagescited by 42
  4. 2023Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generationcited by 33
  5. 2023EverLight: Indoor-Outdoor Editable HDR Lighting Estimationcited by 32
  6. 2024Photorealistic Object Insertion with Diffusion-Guided Inverse Renderingcited by 27
  7. 2024DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenescited by 24
  8. 20253D Scene Generation: A Surveycited by 24
  9. 2025DiT360: High-Fidelity Panoramic Image Generation via Hybrid Trainingcited by 24
  10. 2024PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splattingcited by 23
  11. 2025A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Visioncited by 23
  12. 2023Spatiotemporally Consistent HDR Indoor Lighting Estimationcited by 12
20 more
  1. 2024Taming Stable Diffusion for Text to 360deg Panorama Image Generationcited by 12
  2. 2025DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpaintingcited by 11
  3. 2025Conditional Panoramic Image Generation via Masked Autoregressive Modelingcited by 9
  4. 2025GCC: Generative Color Constancy via Diffusing a Color Checkercited by 8
  5. 2024OPa-Ma: Text Guided Mamba for 360-degree Image Out-paintingcited by 7
  6. 2026360Anything: Geometry-Free Lifting of Images and Videos to 360degcited by 7
  7. 2023Diffusion Posterior Illumination for Ambiguity-aware Inverse Renderingcited by 6
  8. 2024OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generationcited by 5
  9. 2023Scene-Conditional 3D Object Stylization and Compositioncited by 4
  10. 20242S-ODIS: Two-Stage Omni-Directional Image Synthesis by Geometric Distortion Correctioncited by 4
  11. 2025OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenescited by 4
  12. 2022HORIZON: High-Resolution Semantically Controlled Panorama Synthesiscited by 3
  13. 2024360U-Former: HDR Illumination Estimation with Panoramic Adapted Vision Transformerscited by 3
  14. 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scalecited by 2
  15. 2026MAGIC: Transition-Aware Generation of Navigable Multi-Scene Game Worlds with Large Language Modelscited by 1
  16. 2025A Survey on Text-Driven 360-Degree Panorama Generationcited by 0
  17. 2025HDR Environment Map Estimation with Latent Diffusion Modelscited by 0
  18. 2025Top2Pano: Learning to Generate Indoor Panoramas from Top-Down Viewcited by 0
  19. 2025JoPano: Unified Panorama Generation via Joint Modelingcited by 0
  20. 2026StreetDiff: Multi-view Street Scenes Generation via Cross-view Consistent Multi-view Stable Diffusion with Structure Promptscited by 0