# OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenes

arXiv: https://arxiv.org/abs/2510.26800 · HTML: https://mlsanity.com/paper/2510.26800/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 82 references that are in the map.
- First posted on arXiv 30 Oct 2025.

## Builds on (most cited first, 30 of 61)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervision](https://mlsanity.com/paper/2410.19115/) (2024)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion](https://mlsanity.com/paper/2307.01097/) (2023)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [WonderWorld: Interactive 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2406.09394/) (2024)
- [MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyond](https://mlsanity.com/paper/2309.16553/) (2023)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion models](https://mlsanity.com/paper/2405.00666/) (2024)
- [WonderJourney: Going from Anywhere to Everywhere](https://mlsanity.com/paper/2312.03884/) (2023)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [SPEC: Seeing People in the Wild with an Estimated Camera](https://mlsanity.com/paper/2110.00620/) (2021)
- [Rethinking Inductive Biases for Surface Normal Estimation](https://mlsanity.com/paper/2403.00712/) (2024)
- [Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields](https://mlsanity.com/paper/2305.11588/) (2023)
- [3D Common Corruptions and Data Augmentation](https://mlsanity.com/paper/2203.01441/) (2022)

## Built on by (most cited first, 4 of 4)

- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [SonoWorld: From One Image to a 3D Audio-Visual Scene](https://mlsanity.com/paper/2603.28757/) (2026)
- [Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Models](https://mlsanity.com/paper/2604.10578/) (2026)
- [Code Plans, Diffusion Renders: Open-Ended Generative World Modeling](https://mlsanity.com/paper/2609.26458/) (2026)

## Abstract (arXiv)

There are two prevalent ways for automatic 3D scene construction: procedural generation and 2D lifting. Among these, panorama-based 2D lifting has emerged as a promising technique, leveraging powerful 2D generative priors to produce immersive, realistic, and diverse 3D environments. In this work, we advance this technique to generate graphics-ready 3D scenes suitable for physically based rendering (PBR), relighting, and simulation. Our key insight is to repurpose 2D generative models for panorama perception of geometry, textures, and PBR materials. Unlike existing 2D lifting approaches that emphasize appearance generation and neglect the perception of intrinsic properties, we present OmniX, a versatile and unified framework for panorama generation, perception, and completion. Built upon cross-modal adapter structure and cyclic spatial operators, OmniX effectively repurposes pre-trained 2D flow matching priors for joint modeling of multimodal, seamless equirectangular representations. Furthermore, we construct a large-scale synthetic panorama dataset comprising high-quality multimodal panoramas from diverse indoor and outdoor scenes. Extensive experiments demonstrate the effectiveness and generality of OmniX as a unified framework for panorama generation and perception across geometry, lighting, and semantics, enabling graphics-ready 3D scene generation and opening new possibilities for immersive and physically realistic virtual world creation.