mlsanityData to 7 Oct 2026

Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wild

arXiv 2111.12728 ↗First posted 24 Nov 2021cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Tracking and reconstructing 3D objects from cluttered scenes are the key components for computer vision, robotics and autonomous driving systems. While recent progress in implicit function has shown encouraging results on high-quality 3D shape reconstruction, it is still very challenging to generalize to cluttered and partially observable LiDAR data. In this paper, we propose to leverage the continuity in video data. We introduce a novel and unified framework which utilizes a neural implicit function to simultaneously track and reconstruct 3D objects in the wild. Our approach adapts the DeepSDF model (i.e., an instantiation of the implicit function) in the video online, iteratively improving the shape reconstruction while in return improving the tracking, and vice versa. We experiment with both Waymo and KITTI datasets and show significant improvements over state-of-the-art methods for both tracking and shape reconstruction tasks. Our project page is at https://jianglongye.com/implicit-tracking .

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 49 of its 65 references that are in the map. First posted on arXiv 24 Nov 2021.

Builds on

49 of 65 references.

References · 49

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  7. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2020Implicit Geometric Regularization for Learning Shapescited by 707
37 more
  1. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  2. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  3. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  4. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  5. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  6. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 409
  7. 2018PCN: Point Completion Networkcited by 382
  8. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  9. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  10. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  11. 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
  12. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  13. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  14. 2019Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstructioncited by 106
  15. 2017Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular imagecited by 95
  16. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  17. 2020P2B: Point-to-Box Network for 3D Object Tracking in Point Cloudscited by 84
  18. 2018Joint Monocular 3D Vehicle Detection and Trackingcited by 77
  19. 2019Leveraging Shape Completion for 3D Siamese Trackingcited by 74
  20. 2019Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priorscited by 74
  21. 2021Box-Aware Feature Enhancement for Single Object Tracking on Point Cloudscited by 66
  22. 2018ApolloCar3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Drivingcited by 59
  23. 20213D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloudcited by 45
  24. 2020JRMOT: A Real-Time 3D Multi-Object Tracker and a New Large-Scale Datasetcited by 39
  25. 2019Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Cloudscited by 35
  26. 2019FANTrack: 3D Multi-Object Tracking with Feature Association Networkcited by 28
  27. 2020GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervisioncited by 28
  28. 2020DOPS: Learning to Detect 3D Objects and Predict their 3D Shapescited by 27
  29. 2021Model-free Vehicle Tracking and State Estimation in Point Cloud Sequencescited by 25
  30. 2017FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformationcited by 24
  31. 2020SAMP: Shape and Motion Priors for 4D Vehicle Reconstructioncited by 19
  32. 2021Mending Neural Implicit Modeling for 3D Vehicle Reconstruction in the Wildcited by 19
  33. 2019Efficient Bird Eye View Proposals for 3D Siamese Trackingcited by 18
  34. 2019AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objectscited by 12
  35. 2020Semantic Implicit Neural Scene Representations With Semi-Supervised Trainingcited by 9
  36. 2021Localization and Mapping using Instance-specific Mesh Modelscited by 4
  37. 2021Pose Estimation and 3D Reconstruction of Vehicles from Stereo-Images Using a Subcategory-Aware Shape Priorcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 1.

6 papers.

  1. 2023MV-DeepSDF: Implicit Modeling with Multi-Sweep Point Clouds for 3D Vehicle Reconstruction in Autonomous Drivingcited by 11
  2. 2022SpOT: Spatiotemporal Modeling for 3D Object Trackingcited by 10
  3. 2022City-scale Incremental Neural Mapping with Three-layer Sampling and Panoptic Representationcited by 7
  4. 2023LabelFormer: Object Trajectory Refinement for Offboard Perception from LiDAR Point Cloudscited by 7
  5. 2024Pushing Auto-regressive Models for 3D Shape Generation at Capacity and Scalabilitycited by 7
  6. 2023Learning Versatile 3D Shape Generation with Improved AR Modelscited by 3