OccAny: Generalized Unconstrained Urban 3D Occupancy
Not on the map. Why?
Abstract (arXiv)
Relying on in-domain annotations and precise sensor-rig priors, existing 3D occupancy prediction methods are limited in both scalability and out-of-domain generalization. While recent visual geometry foundation models exhibit strong generalization capabilities, they were mainly designed for general purposes and lack one or more key ingredients required for urban occupancy prediction, namely metric prediction, geometry completion in cluttered scenes and adaptation to urban scenarios. We address this gap and present OccAny, the first unconstrained urban 3D occupancy model capable of operating on out-of-domain uncalibrated scenes to predict and complete metric occupancy coupled with segmentation features. OccAny is versatile and can predict occupancy from sequential, monocular, or surround-view images. Our contributions are three-fold: (i) we propose the first generalized 3D occupancy framework with (ii) Segmentation Forcing that improves occupancy quality while enabling mask-level prediction, and (iii) a Novel View Rendering pipeline that infers novel-view geometry to enable test-time view augmentation for geometry completion. Extensive experiments demonstrate that OccAny outperforms all visual geometry baselines on 3D occupancy prediction task, while remaining competitive with in-domain self-supervised methods across three input settings on two established urban occupancy prediction datasets. Our code is available at https://github.com/valeoai/OccAny .
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 77 of its 90 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
77 of 90 references.
References · 77
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
- 2024Depth Anything V2cited by 1,385
- 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
38 more
- 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
- 2024Grounding Image Matching in 3D with MASt3Rcited by 960
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
- 2025Continuous 3D Perception Model with Persistent Statecited by 527
- 2016Semantic Scene Completion from a Single Depth Imagecited by 468
- 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
- 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
- 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
- 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
- 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
- 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
- 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 356
- 2020Virtual KITTI 2cited by 345
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
- 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
- 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
- 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
- 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
- 20243D Reconstruction with Spatial Memorycited by 269
- 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
- 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
- 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 187
- 2021PandaSet: Advanced Sensor Suite Dataset for Autonomous Drivingcited by 186
- 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 155
- 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
- 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
- 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
- 2023NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Spacecited by 88
- 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
- 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 76
- 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
- 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 69
- 2023Behind the Scenes: Density Fields for Single View Reconstructioncited by 63
- 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
50 most cited of 77 All 77 (JSON)
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Height-Guided Projection Reparameterization for Camera-LiDAR Occupancycited by 0
- 2026VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matchingcited by 0
- 2026InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecastingcited by 0
- 2026OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Predictioncited by 0
- 2026How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Modelscited by 0
- 2026Exploring 2D backbone effects for indoor semantic occupancy predictioncited by 0
- 2026Exploring 2D backbone effects for indoor semantic occupancy prediction15 Sep 2026
- 2026How Far Can 5,500 Hours of Driving Take You? A Scaling Law Analysis of Video Diffusion Models28 Aug 2026
- 2026OccAnyScene: Towards Unified Indoor-Outdoor 3D Occupancy Prediction9 Aug 2026
- 2026InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting27 Jul 2026
- 2026VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching29 May 2026
- 2026Height-Guided Projection Reparameterization for Camera-LiDAR Occupancy6 May 2026