mlsanityData to 14 Oct 2026

MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion

arXiv 2306.17000 ↗First posted 29 Jun 2023cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Multiple Object Tracking (MOT) is crucial to autonomous vehicle perception. End-to-end transformer-based algorithms, which detect and track objects simultaneously, show great potential for the MOT task. However, most existing methods focus on image-based tracking with a single object category. In this paper, we propose an end-to-end transformer-based MOT algorithm (MotionTrack) with multi-modality sensor inputs to track objects with multiple classes. Our objective is to establish a transformer baseline for the MOT in an autonomous driving environment. The proposed algorithm consists of a transformer-based data association (DA) module and a transformer-based query enhancement module to achieve MOT and Multiple Object Detection (MOD) simultaneously. The MotionTrack and its variations achieve better results (AMOTA score at 0.55) on the nuScenes dataset compared with other classical baseline models, such as the AB3DMOT, the CenterTrack, and the probabilistic 3D Kalman filter. In addition, we prove that a modified attention mechanism can be utilized for DA to accomplish the MOT, and aggregate history features to enhance the MOD performance.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 46 of its 53 references that are in the map. First posted on arXiv 29 Jun 2023.

Builds on

46 of 53 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  5. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  6. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  7. 2016Simple Online and Realtime Trackingcited by 868
  8. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  9. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  10. 2020Tracking Objects as Pointscited by 483
  11. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  12. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
34 more
  1. 2019Tracking without bells and whistlescited by 382
  2. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  3. 2020TransTrack: Multiple Object Tracking with Transformercited by 371
  4. 2020MOT20: A benchmark for multi object tracking in crowded scenescited by 357
  5. 2019Towards Real-Time Multi-Object Trackingcited by 311
  6. 2015Learning Multi-Domain Convolutional Neural Networks for Visual Trackingcited by 209
  7. 2021Learning to Track with Object Permanencecited by 119
  8. 2022MeMOT: Multi-Object Tracking with Memorycited by 114
  9. 2022Global Tracking Transformerscited by 110
  10. 2021SimpleTrack: Understanding and Rethinking 3D Multi-object Trackingcited by 103
  11. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 91
  12. 2021EagerMOT: 3D Multi-Object Tracking via Sensor Fusioncited by 83
  13. 2021Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Drivingcited by 73
  14. 2020AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metricscited by 63
  15. 2021Exploring Simple 3D Multi-Object Tracking for Autonomous Drivingcited by 56
  16. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  17. 2022CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusioncited by 50
  18. 2017Deep Network Flow for Multi-Object Trackingcited by 45
  19. 2021TransCenter: Transformers with Dense Representations for Multiple-Object Trackingcited by 42
  20. 2021Immortal Tracker: Tracklet Never Diescited by 42
  21. 2018Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Trackingcited by 40
  22. 2018Mono-Camera 3D Multi-Object Tracking Using Deep Learning Detections and PMBM Filteringcited by 32
  23. 2021Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Drivingcited by 26
  24. 2021Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformerscited by 21
  25. 2016Extended Object Tracking: Introduction, Overview and Applicationscited by 16
  26. 2022Transformers for Multi-Object Tracking on Point Cloudscited by 14
  27. 2023Surround-View Vision-based 3D Detection for Autonomous Driving: A Surveycited by 14
  28. 2023Transformer-Based Sensor Fusion for Autonomous Driving: A Surveycited by 14
  29. 2022In Pursuit of Many: A Review of Modern Multiple Object Tracking Systemscited by 12
  30. 2022A Quality Index Metric and Method for Online Self-Assessment of Autonomous Vehicles Sensory Perceptioncited by 10
  31. 2021Attention-based Neural Network for Driving Environment Complexity Perceptioncited by 9
  32. 2022Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object Detection and Trackingcited by 9
  33. 2022Number-Adaptive Prototype Learning for 3D Point Cloud Semantic Segmentationcited by 7
  34. 2021Joint 3D Object Detection and Tracking Using Spatio-Temporal Representation of Camera Image and LiDAR Point Cloudscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 6; 2026: 7.

19 papers.

Sort:

By area:

  1. 2024Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challengecited by 9
  2. 2024Advancing Chart Question Answering with Robust Chart Component Recognitioncited by 5
  3. 20233D Multiple Object Tracking on Autonomous Driving: A Literature Reviewcited by 4
  4. 2024JDT3D: Addressing the Gaps in LiDAR-Based Tracking-by-Attentioncited by 4
  5. 2024LiDAR-based End-to-end Temporal Perception for Vehicle-Infrastructure Cooperationcited by 4
  6. 2024Image-Guided Outdoor LiDAR Perception Quality Assessment for Autonomous Drivingcited by 3
  7. 2025LiDAR MOT-DETR: A LiDAR-based Two-Stage Transformer for 3D Multiple Object Trackingcited by 2
  8. 2025Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Cloudscited by 2
  9. 2025nuScenes Revisited: Progress and Challenges in Autonomous Drivingcited by 2
  10. 2026Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Trackingcited by 2
  11. 2026SCATR: Mitigating New Instance Suppression in LiDAR-based Tracking-by-Attention via Second Chance Assignment and Track Query Dropoutcited by 2
  12. 2025OptiPMB: Enhancing 3D Multi-Object Tracking with Optimized Poisson Multi-Bernoulli Filteringcited by 1
7 more
  1. 2026Fusion-Poly: A Polyhedral Framework Based on Spatial-Temporal Fusion for 3D Multi-Object Trackingcited by 1
  2. 2026Memory-Augmented Vision-Language Agents for Persistent and Semantically Consistent Object Captioningcited by 1
  3. 2026Radar-Informed 3D Multi-Object Tracking under Adverse Conditionscited by 1
  4. 2025FutrTrack: A Camera-LiDAR Fusion Transformer for 3D Multiple Object Trackingcited by 0
  5. 2025End-to-End 3-D Spatiotemporal Perception with Multimodal Fusion and V2X Collaborationcited by 0
  6. 2026GateMOT: Q-Gated Attention for Dense Object Trackingcited by 0
  7. 2026Scene Reconstruction as Mapping Priors for 3D Detectioncited by 0