mlsanityData to 14 Oct 2026

TrackOcc: Camera-based 4D Panoptic Occupancy Tracking

arXiv 2503.08471 ↗First posted 11 Mar 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Comprehensive and consistent dynamic scene understanding from camera input is essential for advanced autonomous systems. Traditional camera-based perception tasks like 3D object tracking and semantic occupancy prediction lack either spatial comprehensiveness or temporal consistency. In this work, we introduce a brand-new task, Camera-based 4D Panoptic Occupancy Tracking, which simultaneously addresses panoptic occupancy segmentation and object tracking from camera-only input. Furthermore, we propose TrackOcc, a cutting-edge approach that processes image inputs in a streaming, end-to-end manner with 4D panoptic queries to address the proposed task. Leveraging the localization-aware loss, TrackOcc enhances the accuracy of 4D panoptic occupancy tracking without bells and whistles. Experimental results demonstrate that our method achieves state-of-the-art performance on the Waymo dataset. The source code will be released at https://github.com/Tsinghua-MARS-Lab/TrackOcc.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

34 of 42 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  6. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  7. 2018Panoptic Segmentationcited by 802
  8. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  9. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  10. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  11. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  12. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
22 more
  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  2. 2022Vision Transformer with Deformable Attentioncited by 295
  3. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 223
  4. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  5. 2019Siam R-CNN: Visual Tracking by Re-Detectioncited by 174
  6. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 165
  7. 2020Video Panoptic Segmentationcited by 142
  8. 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 125
  9. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  10. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 96
  11. 2022MinVIS: A Minimal Video Instance Segmentation Framework without Video-based Trainingcited by 94
  12. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 91
  13. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  14. 2021STEP: Segmenting and Tracking Every Pixelcited by 73
  15. 20214D Panoptic LiDAR Segmentationcited by 65
  16. 2020MOPT: Multi-Object Panoptic Trackingcited by 53
  17. 2023CTVIS: Consistent Training for Online Video Instance Segmentationcited by 51
  18. 2023PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awarenesscited by 42
  19. 20224D-StOP: Panoptic Segmentation of 4D LiDAR using Spatio-temporal Object Proposal Generation and Aggregationcited by 35
  20. 2024ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Associationcited by 33
  21. 2023Mask4Former: Mask Transformer for 4D Panoptic Segmentationcited by 24
  22. 20234D Panoptic Segmentation as Invariant and Equivariant Field Predictioncited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025GS-Occ3D: Scaling Vision-only Occupancy Reconstruction with Gaussian Splattingcited by 11
  2. 2026Latent Gaussian Splatting for 4D Panoptic Occupancy Trackingcited by 3
  3. 2026Streaming Gaussian Encoding for 4D Panoptic Occupancy Trackingcited by 0