mlsanityData to 14 Oct 2026

PRISM: Probabilistic Real-Time Inference in Spatial World Models

arXiv 2212.02988 ↗First posted 6 Dec 2022cs.LG cs.CV cs.ROcited by 0

On the map: World models → World models for navigation

Abstract (arXiv)

We introduce PRISM, a method for real-time filtering in a probabilistic generative model of agent motion and visual perception. Previous approaches either lack uncertainty estimates for the map and agent state, do not run in real-time, do not have a dense scene representation or do not model agent dynamics. Our solution reconciles all of these aspects. We start from a predefined state-space model which combines differentiable rendering and 6-DoF dynamics. Probabilistic inference in this model amounts to simultaneous localisation and mapping (SLAM) and is intractable. We use a series of approximations to Bayesian inference to arrive at probabilistic map and state estimates. We take advantage of well-established methods and closed-form updates, preserving accuracy and enabling real-time capability. The proposed solution runs at 10Hz real-time and is similarly accurate to state-of-the-art SLAM in small to medium-sized indoor environments, with high-speed UAV and handheld camera agents (Blackbird, EuRoC and TUM-RGBD).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 43 of its 92 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

43 of 92 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Tracking and Planning with Spatial World Models25 Jan

Other references · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2016Variational Inference: A Review for Statisticianscited by 1,472
  6. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  7. 2020Neural Sparse Voxel Fieldscited by 959
  8. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  9. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  10. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  11. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 673
  12. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 589
30 more
  1. 2016Direct Sparse Odometrycited by 485
  2. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  3. 2017VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimatorcited by 423
  4. 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
  5. 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 382
  6. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  7. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338
  8. 2020Learning to Explore using Active Neural SLAMcited by 316
  9. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  10. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  11. 2017Cognitive Mapping and Planning for Visual Navigationcited by 212
  12. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 184
  13. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  14. 2015Deep Kalman Filterscited by 166
  15. 2016Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Datacited by 156
  16. 2022iSDF: Real-Time Neural Signed Distance Fields for Robot Perceptioncited by 109
  17. 2017Neural Map: Structured Memory for Deep Reinforcement Learningcited by 90
  18. 2017Recurrent Environment Simulatorscited by 68
  19. 2019Shaping Belief States with Generative Environment Models for RLcited by 66
  20. 2021TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereocited by 39
  21. 2020CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depthcited by 23
  22. 2018Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulationcited by 20
  23. 2020Learning to Fly via Deep Model-Based Reinforcement Learningcited by 20
  24. 2017Unsupervised Real-Time Control through Variational Empowermentcited by 18
  25. 2018PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstructioncited by 13
  26. 2019Incremental Visual-Inertial 3D Mesh Generation with Structural Regularitiescited by 11
  27. 2018Generative Temporal Models with Spatial Memory for Partially Observed Environmentscited by 10
  28. 2018Approximate Bayesian inference in spatial environmentscited by 8
  29. 2022HRBF-Fusion: Accurate 3D reconstruction from RGB-D data using on-the-fly implicitscited by 8
  30. 2020Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoFcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Acting from Belief, Looking When Needed: A Bayesian Spatial World Model for Navigation under Intermittent Perception
  2. 2023Probabilistic Qualitative Localization and Mapping ↗
  3. 2013Occupancy Grids: A Stochastic Spatial Representation for Active Robot Perception
  4. 2026World SLAM Model: Joint World Modeling for SLAM and Navigation
  5. 2026PRISM: PRior-guided Imagination Sampling in world Models
  6. 2026WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigation
  7. 2025Schr"odinger's Navigator: Imagining an Ensemble of Futures for Zero-Shot Object Navigation
  8. 20263D-Belief: Embodied Belief Inference via Generative 3D World Modeling