SPARGen: Unifying Spatial Perception and Reasoning through Native Multimodal Generation
Not on the map. Why?
Abstract (arXiv)
Spatial perception and reasoning from visual observations require recovering geometric structure, establishing correspondences, and understanding spatial relations. Existing approaches typically address these capabilities separately using task-specific architectures or external geometric modules, limiting knowledge transfer among complementary representations of the same physical scene. We introduce SPARGen, a unified multimodal framework that casts 3D reconstruction, dense correspondence, and spatial reasoning as instruction-conditioned generation tasks. SPARGen serializes compact structured and linguistic outputs as token sequences while generating dense geometric fields in image-aligned forms, enabling spatial supervision to jointly shape shared representations within a native multimodal generative model. Experiments across benchmarks for 3D reconstruction, correspondence, and spatial reasoning show that SPARGen achieves competitive performance across heterogeneous spatial tasks within a single native multimodal generative framework.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 14 Aug 2026.
Builds on
30 of 46 references.
References · 30
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
- 2025Continuous 3D Perception Model with Persistent Statecited by 548
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
- 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
18 more
- 2021GMFlow: Learning Optical Flow via Global Matchingcited by 327
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 299
- 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 275
- 2023Visual Geometry Grounded Deep Structure From Motioncited by 227
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 217
- 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 180
- 2022DKM: Dense Kernelized Feature Matching for Geometry Estimationcited by 177
- 2023Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifierscited by 175
- 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 138
- 2024Grounded 3D-LLM with Referent Tokenscited by 115
- 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 106
- 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformercited by 68
- 2025G2VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoningcited by 44
- 2025UFM: A Simple Path towards Unified Dense Correspondence with Flowcited by 36
- 2025OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understandingcited by 24
- 2025UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encodingcited by 10
- 2026Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learningcited by 9
- 2025Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceiverscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence
- 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
- 2026SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning
- 2025Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments
- 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
- 2026GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning