Object-Uni: A Unified Model for Object-Centric Spatial Understanding and Controllable Generation
Not on the map. Why?
Abstract (arXiv)
Unified models for visual understanding and generation have made rapid progress, yet they still lack the ability to understand and manipulate the spatial states of object instances. Existing models can describe objects in natural language, but they struggle to precisely represent continuous object poses and generate geometrically consistent images under target viewpoints. To mitigate this, we propose Object-Uni, a unified model for object-centric spatial understanding and controllable generation. Specifically, we formulate object-centric spatial intelligence as a unified problem connecting pose perception, spatial reasoning, pose-conditioned generation, and object-centric novel view synthesis. We treat object pose as an explicit geometric variable shared by understanding and generation, rather than merely a prediction label or control signal. To make pose usable by multimodal large language models, we propose a viewpoint-based orientation abstraction that maps orientation into structured viewpoint descriptions while preserving continuous geometric supervision. We further construct an object-centric spatial benchmark (UniSpatial-80K) and train a unified model with an object-token-grounded pose anchor to associate each instance with its pose state. Experiments show that our model improves object-level pose understanding and pose-controllable generation, moving unified models from describing objects toward manipulating spatial states.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 28 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
24 of 28 references.
References · 24
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
- 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
12 more
- 2024Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretrainingcited by 148
- 2024Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Modelscited by 45
- 2025SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Modelscited by 37
- 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understandingcited by 25
- 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
- 2026Orient Anything V2: Unifying Orientation and Rotation Understandingcited by 22
- 2024Learning Continuous 3D Words for Text-to-Image Generationcited by 16
- 2024Customizing Text-to-Image Diffusion with Object Viewpoint Controlcited by 13
- 2025Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interactioncited by 13
- 2025Compass Control: Multi Object Orientation Control for Text-to-Image Generationcited by 11
- 2025SceneDesigner: Controllable Multi-Object Image Generation with 9-DoF Pose Manipulationcited by 9
- 2021Scene Designer: a Unified Model for Scene Search and Synthesis from Sketchcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation
- 2025BOP-ASK: Object-Interaction Reasoning for Vision-Language Models
- 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs
- 2025Spatial Reasoning in Foundation Models: Benchmarking Object-Centric Spatial Understanding
- 2025Seeing Isn't Orienting: A Cognitively Informed Hierarchical Benchmark for Object Orientation in MLLMs
- 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation
- 2026Seeing Isn't Orienting: A Cognitively Grounded Benchmark Reveals Systematic Orientation Failures in MLLMs Supplementary
- 2026Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces