# Bridging Language and Spherical Space: Object-Centric Control for Text-to-Panorama Generation

arXiv: https://arxiv.org/abs/2608.20691 · HTML: https://mlsanity.com/paper/2608.20691/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 27 of 27)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023)
- [LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation](https://mlsanity.com/paper/2303.17189/) (2023)
- [InstanceDiffusion: Instance-level Control for Image Generation](https://mlsanity.com/paper/2402.03290/) (2024)
- [MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction](https://mlsanity.com/paper/2402.12712/) (2024)
- [PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigation](https://mlsanity.com/paper/2305.19195/) (2023)
- [Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following](https://mlsanity.com/paper/2311.17002/) (2023)
- [LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation](https://mlsanity.com/paper/2302.08908/) (2023)
- [LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation](https://mlsanity.com/paper/2408.13252/) (2024)
- [HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation](https://mlsanity.com/paper/2410.14324/) (2024)
- [Matrix-3D: Omnidirectional Explorable 3D World Generation](https://mlsanity.com/paper/2508.08086/) (2025)
- [SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation](https://mlsanity.com/paper/2308.10156/) (2023)
- [BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learning](https://mlsanity.com/paper/2112.06179/) (2021)
- [LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts](https://mlsanity.com/paper/2308.06713/) (2023)
- [PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesis](https://mlsanity.com/paper/2403.01852/) (2024)
- [PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Models](https://mlsanity.com/paper/2503.10127/) (2025)
- [InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attention](https://mlsanity.com/paper/2509.16691/) (2025)
- [Conditional Panoramic Image Generation via Masked Autoregressive Modeling](https://mlsanity.com/paper/2505.16862/) (2025)
- [Tacchi 2.0: A Low Computational Cost and Comprehensive Dynamic Contact Simulator for Vision-based Tactile Sensors](https://mlsanity.com/paper/2503.09100/) (2025)
- [CASR: A Robust Cyclic Framework for Arbitrary Large-Scale Super-Resolution with Distribution Alignment and Self-Similarity Awareness](https://mlsanity.com/paper/2602.22159/) (2026)

## Abstract (arXiv)

Panoramic image generation is increasingly important for immersive applications such as virtual reality, augmented reality, and 3D content creation. Unlike perspective images, panoramic images represent a viewer-centered 360^circ surrounding space, where directional expressions such as left, right, front, and behind play a central role in spatial understanding. However, existing text-to-panorama methods largely rely on implicit spatial reasoning and often fail to faithfully ground object-level directional descriptions in spherical panoramic scenes. A straightforward alternative is to introduce explicit layouts, but requiring manually specified spatial conditions reduces the flexibility of language-based interaction and does not directly resolve the misalignment between egocentric directional language and panoramic image space. To address this issue, we propose PanoCtrl, an object-centric framework for controllable text-to-panorama generation. Our method explicitly bridges natural language and spherical panoramic space by converting textual descriptions into structured object-level spherical conditions and integrating them into the diffusion process. Specifically, we introduce PanoParse, a text-conditioned parser that predicts object semantics and spherical bounding field-of-view (BFoV) parameters, and PanoControl, which injects object-level semantic and spatial guidance into the diffusion transformer through object-aware attention and spatial residual enhancement. To support this task, we construct PanoGround, a dataset with object-level spherical annotations and diverse directional descriptions for controllable panoramic generation. Extensive experiments demonstrate that PanoCtrl achieves state-of-the-art performance in both spatial alignment and image quality.