# CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion

arXiv: https://arxiv.org/abs/2209.02540 · HTML: https://mlsanity.com/paper/2209.02540/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 86 references that are in the map.
- First posted on arXiv 6 Sep 2022.

## Builds on (most cited first, 30 of 52)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2203.08195/) (2022)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [PnPNet: End-to-End Perception and Prediction with Tracking in the Loop](https://mlsanity.com/paper/2005.14711/) (2020)
- [Focal Sparse Convolutional Networks for 3D Object Detection](https://mlsanity.com/paper/2204.12463/) (2022)

## Built on by (most cited first, 30 of 50)

- [Dual Radar: A Multi-modal Dataset with Dual 4D Radar for Autonomous Driving](https://mlsanity.com/paper/2310.07602/) (2023)
- [MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddings](https://mlsanity.com/paper/2310.00400/) (2023)
- [V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception](https://mlsanity.com/paper/2411.10962/) (2024)
- [ShaSTA: Modeling Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking](https://mlsanity.com/paper/2211.03919/) (2022)
- [3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking](https://mlsanity.com/paper/2308.06635/) (2023)
- [MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion](https://mlsanity.com/paper/2306.17000/) (2023)
- [MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving](https://mlsanity.com/paper/2409.16149/) (2024)
- [An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2303.12535/) (2023)
- [CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registration](https://mlsanity.com/paper/2309.14660/) (2023)
- [CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perception](https://mlsanity.com/paper/2507.19239/) (2025)
- [Fast-Poly: A Fast Polyhedral Framework For 3D Multi-Object Tracking](https://mlsanity.com/paper/2403.13443/) (2024)
- [ReferGPT: Towards Zero-Shot Referring Multi-Object Tracking](https://mlsanity.com/paper/2504.09195/) (2025)
- [LEGO: Learning and Graph-Optimized Modular Tracker for Online Multi-Object Tracking with Point Clouds](https://mlsanity.com/paper/2308.09908/) (2023)
- [You Only Need Two Detectors to Achieve Multi-Modal 3D Multi-Object Tracking](https://mlsanity.com/paper/2304.08709/) (2023)
- [Collaborative Perception Datasets for Autonomous Driving: A Review](https://mlsanity.com/paper/2504.12696/) (2025)
- [Offline Tracking with Object Permanence](https://mlsanity.com/paper/2310.01288/) (2023)
- [Beyond conventional vision: RGB-event fusion for robust object detection in dynamic traffic scenarios](https://mlsanity.com/paper/2508.10704/) (2025)
- [3D Multiple Object Tracking on Autonomous Driving: A Literature Review](https://mlsanity.com/paper/2309.15411/) (2023)
- [JDT3D: Addressing the Gaps in LiDAR-Based Tracking-by-Attention](https://mlsanity.com/paper/2407.04926/) (2024)
- [Object Re-Identification from Point Clouds](https://mlsanity.com/paper/2305.10210/) (2023)
- [StreamMOTP: Streaming and Unified Framework for Joint 3D Multi-Object Tracking and Trajectory Prediction](https://mlsanity.com/paper/2406.19844/) (2024)
- [A New Architecture for Neural Enhanced Multiobject Tracking](https://mlsanity.com/paper/2410.06294/) (2024)
- [HybridTrack: A Hybrid Approach for Robust Multi-Object Tracking](https://mlsanity.com/paper/2501.01275/) (2025)
- [Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models](https://mlsanity.com/paper/2506.18124/) (2025)
- [Rethinking the Spatio-Temporal Alignment of End-to-End 3D Perception](https://mlsanity.com/paper/2512.23635/) (2025)
- [S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Tracking](https://mlsanity.com/paper/2406.02147/) (2024)
- [RockTrack: A 3D Robust Multi-Camera-Ken Multi-Object Tracking Framework](https://mlsanity.com/paper/2409.11749/) (2024)
- [Towards Next-Generation SLAM: A Survey on 3DGS-SLAM Focusing on Performance, Robustness, and Future Directions](https://mlsanity.com/paper/2602.04251/) (2026)
- [Offline-Poly: A Polyhedral Framework For Offline 3D Multi-Object Tracking](https://mlsanity.com/paper/2602.13772/) (2026)
- [A survey on deep learning approaches for data integration in autonomous driving system](https://mlsanity.com/paper/2306.11740/) (2023)

## Abstract (arXiv)

3D Multi-object tracking (MOT) ensures consistency during continuous dynamic detection, conducive to subsequent motion planning and navigation tasks in autonomous driving. However, camera-based methods suffer in the case of occlusions and it can be challenging to accurately track the irregular motion of objects for LiDAR-based methods. Some fusion methods work well but do not consider the untrustworthy issue of appearance features under occlusion. At the same time, the false detection problem also significantly affects tracking. As such, we propose a novel camera-LiDAR fusion 3D MOT framework based on the Combined Appearance-Motion Optimization (CAMO-MOT), which uses both camera and LiDAR data and significantly reduces tracking failures caused by occlusion and false detection. For occlusion problems, we are the first to propose an occlusion head to select the best object appearance features multiple times effectively, reducing the influence of occlusions. To decrease the impact of false detection in tracking, we design a motion cost matrix based on confidence scores which improve the positioning and object prediction accuracy in 3D space. As existing multi-object tracking methods only consider a single category, we also propose to build a multi-category loss to implement multi-object tracking in multi-category scenes. A series of validation experiments are conducted on the KITTI and nuScenes tracking benchmarks. Our proposed method achieves state-of-the-art performance and the lowest identity switches (IDS) value (23 for Car and 137 for Pedestrian) among all multi-modal MOT methods on the KITTI test dataset. And our proposed method achieves state-of-the-art performance among all algorithms on the nuScenes test dataset with 75.3% AMOTA.