mlsanityData to 14 Oct 2026

PVO: Panoptic Visual Odometry

arXiv 2207.01610 ↗First posted 4 Jul 2022cs.CV cs.ROcited by 26

Not on the map. Why?

Abstract (arXiv)

We present PVO, a novel panoptic visual odometry framework to achieve more comprehensive modeling of the scene motion, geometry, and panoptic segmentation information. Our PVO models visual odometry (VO) and video panoptic segmentation (VPS) in a unified view, which makes the two tasks mutually beneficial. Specifically, we introduce a panoptic update module into the VO Module with the guidance of image panoptic segmentation. This Panoptic-Enhanced VO Module can alleviate the impact of dynamic objects in the camera pose estimation with a panoptic-aware dynamic mask. On the other hand, the VO-Enhanced VPS Module also improves the segmentation accuracy by fusing the panoptic segmentation result of the current frame on the fly to the adjacent frames, using geometric information such as camera pose, depth, and optical flow obtained from the VO Module. These two modules contribute to each other through recurrent iterative optimization. Extensive experiments demonstrate that PVO outperforms state-of-the-art methods in both visual odometry and video panoptic segmentation tasks.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 44 of its 64 references that are in the map. First posted on arXiv 4 Jul 2022.

Builds on

44 of 64 references.

References · 44

  1. 2017Mask R-CNNcited by 7,277
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  3. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  4. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  5. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  6. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  7. 2019Panoptic Feature Pyramid Networkscited by 645
  8. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  9. 2016Direct Sparse Odometrycited by 485
  10. 2019Video Instance Segmentationcited by 393
  11. 2020Virtual KITTI 2cited by 353
  12. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
32 more
  1. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  2. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  3. 2017DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networkscited by 175
  4. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  5. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  6. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  7. 2020Video Panoptic Segmentationcited by 142
  8. 2018DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenescited by 134
  9. 2020TartanVO: A Generalizable Learning-based VOcited by 108
  10. 2021SuMa++: Efficient LiDAR-based Semantic SLAMcited by 100
  11. 2020ViP-DeepLab: Learning Visual Perception with Depth-aware Video Panoptic Segmentationcited by 96
  12. 2018MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objectscited by 95
  13. 2017SegFlow: Joint Learning for Video Object Segmentation and Optical Flowcited by 92
  14. 2020Fully Convolutional Networks for Panoptic Segmentationcited by 92
  15. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  16. 2021STEP: Segmenting and Tracking Every Pixelcited by 73
  17. 2018MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAMcited by 65
  18. 2021Omnimatte: Associating Objects and Their Effects in Videocited by 50
  19. 2020DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAMcited by 47
  20. 2016Video Propagation Networkscited by 45
  21. 2020VDO-SLAM: A Visual Dynamic Object-aware SLAM Systemcited by 40
  22. 2019Every Frame Counts: Joint Learning of Video Segmentation and Optical Flowcited by 31
  23. 2021Learning to Associate Every Segment for Video Panoptic Segmentationcited by 25
  24. 2020EffiScene: Efficient Per-Pixel Rigidity Inference for Unsupervised Joint Learning of Optical Flow, Depth, Camera Pose and Motion Segmentationcited by 22
  25. 2022FD-SLAM: 3-D Reconstruction Using Features and Dense Matchingcited by 17
  26. 2021NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mappingcited by 14
  27. 2021Object-Augmented RGB-D SLAM for Wide-Disparity Relocalisationcited by 7
  28. 2019SimVODIS: Simultaneous Visual Odometry, Object Detection, and Instance Segmentationcited by 6
  29. 2021Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Videocited by 5
  30. 2021S3LAM: Structured Scene SLAMcited by 5
  31. 2022Indoor Navigation Assistance for Visually Impaired People via Dynamic SLAM and Panoptic Segmentation with an RGB-D Sensorcited by 5
  32. 2022Hybrid Tracker with Pixel and Instance for Video Panoptic Segmentationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 14; 2025: 4; 2026: 1.

26 papers.

Sort:

By area:

  1. 2024PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstructioncited by 285
  2. 2024NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wildcited by 85
  3. 2024Deep Patch Visual SLAMcited by 71
  4. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 56
  5. 2022IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesiscited by 50
  6. 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusioncited by 38
  7. 2025CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstructioncited by 26
  8. 2022iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Trackingcited by 22
  9. 2023GP-NeRF: Generalized Perception NeRF for Context-Aware 3D Scene Understandingcited by 20
  10. 2024StreetSurfGS: Scalable Urban Street Surface Reconstruction with Planar-based Gaussian Splattingcited by 19
  11. 2024NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstructioncited by 16
  12. 2024Reinforcement Learning Meets Visual Odometrycited by 14
14 more
  1. 2024ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstructioncited by 11
  2. 2022D3FlowSLAM: Self-Supervised Dynamic SLAM with Flow Motion Decomposition and DINO Guidancecited by 9
  3. 2023DVI-SLAM: A Dual Visual Inertial SLAM Networkcited by 9
  4. 2023RD-VIO: Robust Visual-Inertial Odometry for Mobile Augmented Reality in Dynamic Environmentscited by 9
  5. 2025Large-Scale Gaussian Splatting SLAMcited by 7
  6. 2023MC-NeRF: Multi-Camera Neural Radiance Fields for Multi-Camera Image Acquisition Systemscited by 5
  7. 2024DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wildcited by 4
  8. 2025ZeroVO: Visual Odometry with Minimal Assumptionscited by 4
  9. 2024NGD-SLAM: Towards Real-Time Dynamic SLAM without GPUcited by 2
  10. 2024Self-Supervised Geometry-Guided Initialization for Robust Monocular Visual Odometrycited by 1
  11. 2026MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometrycited by 1
  12. 2024An Earth Rover dataset recorded at the ICRA@40 partycited by 0
  13. 2024STAG-VIO: Stabilized Prompt-to-Geometry Interface for Robust Dynamic Visual--Inertial Odometrycited by 0
  14. 2025SPORTS: Simultaneous Panoptic Odometry, Rendering, Tracking and Segmentation for Urban Scenes Understandingcited by 0