mlsanityData to 7 Oct 2026

TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception

arXiv 2609.32013 ↗First posted 25 Sep 2026cs.CV cs.AIcited by 0

On the map: World models → World modeling from LiDAR generation also fits: World models for autonomous driving

Abstract (arXiv)

We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to prior work, TriO utilizes three distinct sensor modalities (camera, LiDAR, and RADAR) as both inputs and sources of self-supervision, eliminating the need for additional human annotations. Thanks to its novel supervision, the model is able to segment any occupancy from the drivable surface, overcoming the limitations of existing open-set methods in handling long-tail objects. TriO achieves state-of-the-art results in multiple 3D and 4D tasks, including occupancy, flow, and LiDAR prediction, as well as zero-shot road obstacle segmentation across multiple datasets such as Argoverse 2, and Spotting the Unexpected.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World modeling from LiDAR generation, one of 11 directions in World models. Builds on 87 of its 112 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

87 of 112 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks25 Jan

Other references · 86

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023Segment Anythingcited by 8,158
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  8. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  9. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  10. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  11. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  12. 2018PCN: Point Completion Networkcited by 382
38 more
  1. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  2. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  3. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  4. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  5. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  6. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  7. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  8. 2017Submanifold Sparse Convolutional Networkscited by 223
  9. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  10. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 218
  11. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  12. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
  13. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  14. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  15. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  16. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  17. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  18. 2021Neural Scene Flow Priorcited by 91
  19. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 83
  20. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  21. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecastingcited by 76
  22. 20213D Semantic Scene Completion: a Surveycited by 74
  23. 2023UniPAD: A Universal Pre-training Paradigm for Autonomous Drivingcited by 73
  24. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  25. 2022Occupancy Flow Fields for Motion Forecasting in Autonomous Drivingcited by 62
  26. 2024DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Modelcited by 58
  27. 2023Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applicationscited by 50
  28. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  29. 2020Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approachcited by 47
  30. 2024Better Call SAL: Towards Learning to Segment Anything in Lidarcited by 47
  31. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  32. 2022Patchwork++: Fast and Robust Ground Segmentation Solving Partial Under-Segmentation Using 3D Point Cloudcited by 45
  33. 2024Talking to DINO: Bridging Self-Supervised Vision Backbones with Language for Open-Vocabulary Segmentationcited by 42
  34. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecastingcited by 40
  35. 2024Collaborative Vision-Text Representation Optimizing for Open-Vocabulary Segmentationcited by 40
  36. 2022Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Cloudscited by 39
  37. 2023Fast Neural Scene Flowcited by 39
  38. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39

50 most cited of 86 All 86 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GaussianOcc3D: A Gaussian-Based Adaptive Multi-modal 3D Occupancy Prediction
  2. 2025MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategies
  3. 2025OccLE: Label-Efficient 3D Semantic Occupancy Prediction
  4. 2023LiDAR-based 4D Occupancy Completion and Forecasting
  5. 2023UniWorld: Autonomous Driving Pre-training via World Models
  6. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representations
  7. 2025Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving
  8. 2024OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection