Bridging Language and Spherical Space: Object-Centric Control for Text-to-Panorama Generation
Not on the map. Why?
Abstract (arXiv)
Panoramic image generation is increasingly important for immersive applications such as virtual reality, augmented reality, and 3D content creation. Unlike perspective images, panoramic images represent a viewer-centered 360^circ surrounding space, where directional expressions such as left, right, front, and behind play a central role in spatial understanding. However, existing text-to-panorama methods largely rely on implicit spatial reasoning and often fail to faithfully ground object-level directional descriptions in spherical panoramic scenes. A straightforward alternative is to introduce explicit layouts, but requiring manually specified spatial conditions reduces the flexibility of language-based interaction and does not directly resolve the misalignment between egocentric directional language and panoramic image space. To address this issue, we propose PanoCtrl, an object-centric framework for controllable text-to-panorama generation. Our method explicitly bridges natural language and spherical panoramic space by converting textual descriptions into structured object-level spherical conditions and integrating them into the diffusion process. Specifically, we introduce PanoParse, a text-conditioned parser that predicts object semantics and spherical bounding field-of-view (BFoV) parameters, and PanoControl, which injects object-level semantic and spatial guidance into the diffusion transformer through object-aware attention and spatial residual enhancement. To support this task, we construct PanoGround, a dataset with object-level spherical annotations and diverse directional descriptions for controllable panoramic generation. Extensive experiments demonstrate that PanoCtrl achieves state-of-the-art performance in both spatial alignment and image quality.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 21 Aug 2026.
Builds on
27 of 53 references.
References · 27
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2016Improved Techniques for Training GANscited by 2,970
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
- 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 259
- 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
- 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstructioncited by 95
15 more
- 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
- 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Followingcited by 77
- 2023LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generationcited by 65
- 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
- 2024HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generationcited by 41
- 2025Matrix-3D: Omnidirectional Explorable 3D World Generationcited by 35
- 2023SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generationcited by 29
- 2021BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learningcited by 26
- 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layoutscited by 20
- 2024PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesiscited by 20
- 2025PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Modelscited by 18
- 2025InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attentioncited by 10
- 2025Conditional Panoramic Image Generation via Masked Autoregressive Modelingcited by 9
- 2025Tacchi 2.0: A Low Computational Cost and Comprehensive Dynamic Contact Simulator for Vision-based Tactile Sensorscited by 4
- 2026CASR: A Robust Cyclic Framework for Arbitrary Large-Scale Super-Resolution with Distribution Alignment and Self-Similarity Awarenesscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2022HORIZON: High-Resolution Semantically Controlled Panorama Synthesis
- 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion
- 2025PanoGen++: Domain-Adapted Text-Guided Panoramic Environment Generation for Vision-and-Language Navigation
- 2026PSGS: Text-driven Panorama Sliding Scene Generation via Gaussian Splatting
- 2025A Survey on Text-Driven 360-Degree Panorama Generation
- 2025PanoWorld-X: Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusion
- 2026SpheRoPE: Zero-Shot Optimization-Free 360 Panorama Generation with Spherical RoPE
- 2026PanoWorld: Towards Spatial Supersensing in 360^circ Panorama World