# 3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention

arXiv: https://arxiv.org/abs/2203.10926 · HTML: https://mlsanity.com/paper/2203.10926/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [Learning a Neural Solver for Multiple Object Tracking](https://mlsanity.com/paper/1912.07515/) (2019)
- [Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking](https://mlsanity.com/paper/2109.03805/) (2021)
- [CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2011.04841/) (2020)
- [GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning](https://mlsanity.com/paper/2006.07327/) (2020)
- [EagerMOT: 3D Multi-Object Tracking via Sensor Fusion](https://mlsanity.com/paper/2104.14682/) (2021)
- [Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2012.13755/) (2020)
- [Probabilistic 3D Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2001.05673/) (2020)
- [Robust Multi-Modality Multi-Object Tracking](https://mlsanity.com/paper/1909.03850/) (2019)
- [Lifted Disjoint Paths with Application in Multiple Object Tracking](https://mlsanity.com/paper/2006.14550/) (2020)
- [MOPT: Multi-Object Panoptic Tracking](https://mlsanity.com/paper/2004.08189/) (2020)
- [There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge](https://mlsanity.com/paper/2103.01353/) (2021)
- [Learnable Online Graph Representations for 3D Multi-Object Tracking](https://mlsanity.com/paper/2104.11747/) (2021)
- [End-to-end Learning of Multi-sensor 3D Tracking by Detection](https://mlsanity.com/paper/1806.11534/) (2018)
- [Multimodal Interaction-aware Motion Prediction for Autonomous Street Crossing](https://mlsanity.com/paper/1808.06887/) (2018)
- [Vision-Based Autonomous UAV Navigation and Landing for Urban Search and Rescue](https://mlsanity.com/paper/1906.01304/) (2019)
- [Factor Graph based 3D Multi-Object Tracking in Point Clouds](https://mlsanity.com/paper/2008.05309/) (2020)
- [Learning Optimal Parameters for Multi-target Tracking with Contextual Interactions](https://mlsanity.com/paper/1610.01394/) (2016)

## Built on by (most cited first, 27 of 27)

- [Learning and Aggregating Lane Graphs for Urban Automated Driving](https://mlsanity.com/paper/2302.06175/) (2023)
- [Language-Driven Active Learning for Diverse Open-Set 3D Object Detection](https://mlsanity.com/paper/2404.12856/) (2024)
- [AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving](https://mlsanity.com/paper/2309.06547/) (2023)
- [In Pursuit of Many: A Review of Modern Multiple Object Tracking Systems](https://mlsanity.com/paper/2209.04796/) (2022)
- [LEGO: Learning and Graph-Optimized Modular Tracker for Online Multi-Object Tracking with Point Clouds](https://mlsanity.com/paper/2308.09908/) (2023)
- [Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking](https://mlsanity.com/paper/2510.10287/) (2025)
- [Amodal Optical Flow](https://mlsanity.com/paper/2311.07761/) (2023)
- [Offline Tracking with Object Permanence](https://mlsanity.com/paper/2310.01288/) (2023)
- [Evidential Uncertainty Estimation for Multi-Modal Trajectory Prediction](https://mlsanity.com/paper/2503.05274/) (2025)
- [3D Multiple Object Tracking on Autonomous Driving: A Literature Review](https://mlsanity.com/paper/2309.15411/) (2023)
- [Motion Forecasting via Model-Based Risk Minimization](https://mlsanity.com/paper/2409.10585/) (2024)
- [GSLAMOT: A Tracklet and Query Graph-based Simultaneous Locating, Mapping, and Multiple Object Tracking System](https://mlsanity.com/paper/2408.09191/) (2024)
- [A New Architecture for Neural Enhanced Multiobject Tracking](https://mlsanity.com/paper/2410.06294/) (2024)
- [Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models](https://mlsanity.com/paper/2506.18124/) (2025)
- [Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking](https://mlsanity.com/paper/2602.23172/) (2026)
- [Autonomous Apple Fruitlet Sizing and Growth Rate Tracking using Computer Vision](https://mlsanity.com/paper/2212.01506/) (2022)
- [A Hybrid Graph Network for Complex Activity Detection in Video](https://mlsanity.com/paper/2310.17493/) (2023)
- [Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision](https://mlsanity.com/paper/2603.18298/) (2026)
- [Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Tracking](https://mlsanity.com/paper/2604.25405/) (2026)
- [BOTT: Box Only Transformer Tracker for 3D Object Tracking](https://mlsanity.com/paper/2308.08753/) (2023)
- [Contour Errors: Ego-Centric Matching for 3D Multi-Object Tracking Performance Evaluation](https://mlsanity.com/paper/2506.04122/) (2025)
- [GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure](https://mlsanity.com/paper/2608.14428/) (2026)
- [Learning Lane Graphs from Aerial Imagery Using Transformers](https://mlsanity.com/paper/2407.05687/) (2024)
- [CORT: Class-Oriented Real-time Tracking for Embedded Systems](https://mlsanity.com/paper/2407.17521/) (2024)
- [Interpretable Dynamic Graph Neural Networks for Small Occluded Object Detection and Tracking](https://mlsanity.com/paper/2411.17251/) (2024)
- [Visual Loop Closure Detection Through Deep Graph Consensus](https://mlsanity.com/paper/2505.21754/) (2025)
- [Taming Perception Jitter: Uncertainty-Aware LiDAR Object Detection for Reliable Motion Classification](https://mlsanity.com/paper/2606.09350/) (2026)

## Abstract (arXiv)

Online 3D multi-object tracking (MOT) has witnessed significant research interest in recent years, largely driven by demand from the autonomous systems community. However, 3D offline MOT is relatively less explored. Labeling 3D trajectory scene data at a large scale while not relying on high-cost human experts is still an open research question. In this work, we propose Batch3DMOT which follows the tracking-by-detection paradigm and represents real-world scenes as directed, acyclic, and category-disjoint tracking graphs that are attributed using various modalities such as camera, LiDAR, and radar. We present a multi-modal graph neural network that uses a cross-edge attention mechanism mitigating modality intermittence, which translates into sparsity in the graph domain. Additionally, we present attention-weighted convolutions over frame-wise k-NN neighborhoods as suitable means to allow information exchange across disconnected graph components. We evaluate our approach using various sensor modalities and model configurations on the challenging nuScenes and KITTI datasets. Extensive experiments demonstrate that our proposed approach yields an overall improvement of 3.3% in the AMOTA score on nuScenes thereby setting the new state-of-the-art for 3D tracking and further enhancing false positive filtering.