# Guided Co-Modulated GAN for 360deg Field of View Extrapolation

arXiv: https://arxiv.org/abs/2204.07286 · HTML: https://mlsanity.com/paper/2204.07286/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 80 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 30 of 62)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Generative Visual Manipulation on the Natural Image Manifold](https://mlsanity.com/paper/1609.03552/) (2016)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image](https://mlsanity.com/paper/2012.09855/) (2020)
- [DIODE: A Dense Indoor and Outdoor DEpth Dataset](https://mlsanity.com/paper/1908.00463/) (2019)

## Built on by (most cited first, 30 of 32)

- [DiffusionLight: Light Probes for Free by Painting a Chrome Ball](https://mlsanity.com/paper/2312.09168/) (2023)
- [360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model](https://mlsanity.com/paper/2401.06578/) (2024)
- [360-Degree Panorama Generation from Few Unregistered NFoV Images](https://mlsanity.com/paper/2308.14686/) (2023)
- [Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation](https://mlsanity.com/paper/2309.03467/) (2023)
- [EverLight: Indoor-Outdoor Editable HDR Lighting Estimation](https://mlsanity.com/paper/2304.13207/) (2023)
- [Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering](https://mlsanity.com/paper/2408.09702/) (2024)
- [DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes](https://mlsanity.com/paper/2412.11100/) (2024)
- [PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting](https://mlsanity.com/paper/2412.12096/) (2024)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training](https://mlsanity.com/paper/2510.11712/) (2025)
- [Spatiotemporally Consistent HDR Indoor Lighting Estimation](https://mlsanity.com/paper/2305.04374/) (2023)
- [Taming Stable Diffusion for Text to 360deg Panorama Image Generation](https://mlsanity.com/paper/2404.07949/) (2024)
- [DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting](https://mlsanity.com/paper/2507.01305/) (2025)
- [Conditional Panoramic Image Generation via Masked Autoregressive Modeling](https://mlsanity.com/paper/2505.16862/) (2025)
- [GCC: Generative Color Constancy via Diffusing a Color Checker](https://mlsanity.com/paper/2502.17435/) (2025)
- [OPa-Ma: Text Guided Mamba for 360-degree Image Out-painting](https://mlsanity.com/paper/2407.10923/) (2024)
- [360Anything: Geometry-Free Lifting of Images and Videos to 360deg](https://mlsanity.com/paper/2601.16192/) (2026)
- [Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering](https://mlsanity.com/paper/2310.00362/) (2023)
- [OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation](https://mlsanity.com/paper/2412.09623/) (2024)
- [Scene-Conditional 3D Object Stylization and Composition](https://mlsanity.com/paper/2312.12419/) (2023)
- [2S-ODIS: Two-Stage Omni-Directional Image Synthesis by Geometric Distortion Correction](https://mlsanity.com/paper/2409.09969/) (2024)
- [OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenes](https://mlsanity.com/paper/2510.26800/) (2025)
- [HORIZON: High-Resolution Semantically Controlled Panorama Synthesis](https://mlsanity.com/paper/2210.04522/) (2022)
- [360U-Former: HDR Illumination Estimation with Panoramic Adapted Vision Transformers](https://mlsanity.com/paper/2410.13566/) (2024)
- [UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scale](https://mlsanity.com/paper/2511.18005/) (2025)
- [MAGIC: Transition-Aware Generation of Navigable Multi-Scene Game Worlds with Large Language Models](https://mlsanity.com/paper/2607.11594/) (2026)
- [A Survey on Text-Driven 360-Degree Panorama Generation](https://mlsanity.com/paper/2502.14799/) (2025)
- [HDR Environment Map Estimation with Latent Diffusion Models](https://mlsanity.com/paper/2507.21261/) (2025)
- [Top2Pano: Learning to Generate Indoor Panoramas from Top-Down View](https://mlsanity.com/paper/2507.21371/) (2025)

## Abstract (arXiv)

We propose a method to extrapolate a 360deg field of view from a single image that allows for user-controlled synthesis of the out-painted content. To do so, we propose improvements to an existing GAN-based in-painting architecture for out-painting panoramic image representation. Our method obtains state-of-the-art results and outperforms previous methods on standard image quality metrics. To allow controlled synthesis of out-painting, we introduce a novel guided co-modulation framework, which drives the image generation process with a common pretrained discriminative model. Doing so maintains the high visual quality of generated panoramas while enabling user-controlled semantic content in the extrapolated field of view. We demonstrate the state-of-the-art results of our method on field of view extrapolation both qualitatively and quantitatively, providing thorough analysis of our novel editing capabilities. Finally, we demonstrate that our approach benefits the photorealistic virtual insertion of highly glossy objects in photographs.