# Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation

arXiv: https://arxiv.org/abs/2309.03467 · HTML: https://mlsanity.com/paper/2309.03467/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 41 references that are in the map.
- First posted on arXiv 7 Sep 2023.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Scaling up GANs for Text-to-Image Synthesis](https://mlsanity.com/paper/2303.05511/) (2023)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [LDM3D: Latent Diffusion Model for 3D](https://mlsanity.com/paper/2305.10853/) (2023)
- [HDR Environment Map Estimation for Real-Time Augmented Reality](https://mlsanity.com/paper/2011.10687/) (2020)
- [Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creation](https://mlsanity.com/paper/2203.14668/) (2022)
- [Learning High Dynamic Range from Outdoor Panoramas](https://mlsanity.com/paper/1703.10200/) (2017)
- [Guided Co-Modulated GAN for 360deg Field of View Extrapolation](https://mlsanity.com/paper/2204.07286/) (2022)
- [Painting Outside the Box: Image Outpainting with GANs](https://mlsanity.com/paper/1808.08483/) (2018)
- [AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation](https://mlsanity.com/paper/2305.13050/) (2023)
- [Image Outpainting and Harmonization using Generative Adversarial Networks](https://mlsanity.com/paper/1912.10960/) (2019)
- [Bridging the Visual Gap: Wide-Range Image Blending](https://mlsanity.com/paper/2103.15149/) (2021)

## Built on by (most cited first, 30 of 33)

- [DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion](https://mlsanity.com/paper/2410.24203/) (2024)
- [CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation](https://mlsanity.com/paper/2501.17162/) (2025)
- [WonderTurbo: Generating Interactive 3D World in 0.72 Seconds](https://mlsanity.com/paper/2504.02261/) (2025)
- [Imagine360: Immersive 360 Video Generation from Perspective Anchor](https://mlsanity.com/paper/2412.03552/) (2024)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [DiT360: High-Fidelity Panoramic Image Generation via Hybrid Training](https://mlsanity.com/paper/2510.11712/) (2025)
- [What Makes for Text to 360-degree Panorama Generation with Stable Diffusion?](https://mlsanity.com/paper/2505.22129/) (2025)
- [Taming Stable Diffusion for Text to 360deg Panorama Image Generation](https://mlsanity.com/paper/2404.07949/) (2024)
- [Panorama Generation From NFoV Image Done Right](https://mlsanity.com/paper/2503.18420/) (2025)
- [CubeComposer: Spatio-Temporal Autoregressive 4K 360deg Video Generation from Perspective Video](https://mlsanity.com/paper/2603.04291/) (2026)
- [Geometry Fidelity for Spherical Images](https://mlsanity.com/paper/2407.18207/) (2024)
- [Conditional Panoramic Image Generation via Masked Autoregressive Modeling](https://mlsanity.com/paper/2505.16862/) (2025)
- [OPa-Ma: Text Guided Mamba for 360-degree Image Out-painting](https://mlsanity.com/paper/2407.10923/) (2024)
- [360Anything: Geometry-Free Lifting of Images and Videos to 360deg](https://mlsanity.com/paper/2601.16192/) (2026)
- [B-VLLM: A Vision Large Language Model with Balanced Spatio-Temporal Tokens](https://mlsanity.com/paper/2412.09919/) (2024)
- [Latent Swap Joint Diffusion for 2D Long-Form Latent Generation](https://mlsanity.com/paper/2502.05130/) (2025)
- [SPGen: Spherical Projection as Consistent and Flexible Representation for Single Image 3D Shape Generation](https://mlsanity.com/paper/2509.12721/) (2025)
- [360PanT: Training-Free Text-Driven 360-Degree Panorama-to-Panorama Translation](https://mlsanity.com/paper/2409.08397/) (2024)
- [2S-ODIS: Two-Stage Omni-Directional Image Synthesis by Geometric Distortion Correction](https://mlsanity.com/paper/2409.09969/) (2024)
- [DuoCast: Duo-Probabilistic Diffusion for Precipitation Nowcasting](https://mlsanity.com/paper/2412.01091/) (2024)
- [GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction](https://mlsanity.com/paper/2512.25073/) (2025)
- [Radio Frequency Signal based Human Silhouette Segmentation: A Sequential Diffusion Approach](https://mlsanity.com/paper/2407.19244/) (2024)
- [Fuzzing: Randomness? Reasoning! Efficient Directed Fuzzing via Large Language Models](https://mlsanity.com/paper/2507.22065/) (2025)
- [SE360: Semantic Edit in 360^circ Panoramas via Hierarchical Data Construction](https://mlsanity.com/paper/2512.19943/) (2025)
- [Text2Sign Diffusion: A Generative Approach for Gloss-Free Sign Language Production](https://mlsanity.com/paper/2509.10845/) (2025)
- [A Survey on Text-Driven 360-Degree Panorama Generation](https://mlsanity.com/paper/2502.14799/) (2025)
- [HDR Environment Map Estimation with Latent Diffusion Models](https://mlsanity.com/paper/2507.21261/) (2025)
- [JoPano: Unified Panorama Generation via Joint Modeling](https://mlsanity.com/paper/2512.06885/) (2025)
- [Stepper: Stepwise Immersive Scene Generation with Multiview Panoramas](https://mlsanity.com/paper/2603.28980/) (2026)

## Abstract (arXiv)

A 360-degree (omni-directional) image provides an all-encompassing spherical view of a scene. Recently, there has been an increasing interest in synthesising 360-degree images from conventional narrow field of view (NFoV) images captured by digital cameras and smartphones, for providing immersive experiences in various scenarios such as virtual reality. Yet, existing methods typically fall short in synthesizing intricate visual details or ensure the generated images align consistently with user-provided prompts. In this study, autoregressive omni-aware generative network (AOG-Net) is proposed for 360-degree image generation by out-painting an incomplete 360-degree image progressively with NFoV and text guidances joinly or individually. This autoregressive scheme not only allows for deriving finer-grained and text-consistent patterns by dynamically generating and adjusting the process but also offers users greater flexibility to edit their conditions throughout the generation process. A global-local conditioning mechanism is devised to comprehensively formulate the outpainting guidance in each autoregressive step. Text guidances, omni-visual cues, NFoV inputs and omni-geometry are encoded and further formulated with cross-attention based transformers into a global stream and a local stream into a conditioned generative backbone model. As AOG-Net is compatible to leverage large-scale models for the conditional encoder and the generative prior, it enables the generation to use extensive open-vocabulary text guidances. Comprehensive experiments on two commonly used 360-degree image datasets for both indoor and outdoor settings demonstrate the state-of-the-art performance of our proposed method. Our code will be made publicly available.