# GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning

arXiv: https://arxiv.org/abs/2006.07327 · HTML: https://mlsanity.com/paper/2006.07327/

## Key facts

- Cited by 113 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 30 of 113)

- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking](https://mlsanity.com/paper/2111.09621/) (2021)
- [MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries](https://mlsanity.com/paper/2205.00613/) (2022)
- [EagerMOT: 3D Multi-Object Tracking via Sensor Fusion](https://mlsanity.com/paper/2104.14682/) (2021)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [Unifying Short and Long-Term Tracking with Graph Hierarchies](https://mlsanity.com/paper/2212.03038/) (2022)
- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking](https://mlsanity.com/paper/2302.03802/) (2023)
- [Graph-based Topology Reasoning for Driving Scenes](https://mlsanity.com/paper/2304.05277/) (2023)
- [Graph Filters for Signal Processing and Machine Learning on Graphs](https://mlsanity.com/paper/2211.08854/) (2022)
- [RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation](https://mlsanity.com/paper/2105.04322/) (2021)
- [Learnable Graph Matching: Incorporating Graph Partitioning with Deep Feature Learning for Multiple Object Tracking](https://mlsanity.com/paper/2103.16178/) (2021)
- [3D Siamese Transformer Network for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2207.11995/) (2022)
- [TransCenter: Transformers with Dense Representations for Multiple-Object Tracking](https://mlsanity.com/paper/2103.15145/) (2021)
- [Immortal Tracker: Tracklet Never Dies](https://mlsanity.com/paper/2111.13672/) (2021)
- [Learnable Online Graph Representations for 3D Multi-Object Tracking](https://mlsanity.com/paper/2104.11747/) (2021)
- [Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision](https://mlsanity.com/paper/2303.00462/) (2023)
- [DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association](https://mlsanity.com/paper/2202.12100/) (2022)
- [Score refinement for confidence-based 3D multi-object tracking](https://mlsanity.com/paper/2107.04327/) (2021)
- [PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object Tracking?](https://mlsanity.com/paper/2208.01957/) (2022)
- [Quo Vadis: Is Trajectory Forecasting the Key Towards Long-Term Multi-Object Tracking?](https://mlsanity.com/paper/2210.07681/) (2022)
- [Forecasting from LiDAR via Future Object Detection](https://mlsanity.com/paper/2203.16297/) (2022)
- [Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection](https://mlsanity.com/paper/2304.12315/) (2023)
- [3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention](https://mlsanity.com/paper/2203.10926/) (2022)
- [Track without Appearance: Learn Box and Tracklet Embedding with Local and Global Motion Patterns for Vehicle Tracking](https://mlsanity.com/paper/2108.06029/) (2021)
- [TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM](https://mlsanity.com/paper/2210.16204/) (2022)
- [Fooling LiDAR Perception via Adversarial Trajectory Perturbation](https://mlsanity.com/paper/2103.15326/) (2021)
- [GraphAD: Interaction Scene Graph for End-to-end Autonomous Driving](https://mlsanity.com/paper/2403.19098/) (2024)
- [A Lightweight and Detector-free 3D Single Object Tracker on Point Clouds](https://mlsanity.com/paper/2203.04232/) (2022)
- [ODAM: Object Detection, Association, and Mapping using Posed RGB Video](https://mlsanity.com/paper/2108.10165/) (2021)

## Abstract (arXiv)

3D Multi-object tracking (MOT) is crucial to autonomous systems. Recent work uses a standard tracking-by-detection pipeline, where feature extraction is first performed independently for each object in order to compute an affinity matrix. Then the affinity matrix is passed to the Hungarian algorithm for data association. A key process of this standard pipeline is to learn discriminative features for different objects in order to reduce confusion during data association. In this work, we propose two techniques to improve the discriminative feature learning for MOT: (1) instead of obtaining features for each object independently, we propose a novel feature interaction mechanism by introducing the Graph Neural Network. As a result, the feature of one object is informed of the features of other objects so that the object feature can lean towards the object with similar feature (i.e., object probably with a same ID) and deviate from objects with dissimilar features (i.e., object probably with different IDs), leading to a more discriminative feature for each object; (2) instead of obtaining the feature from either 2D or 3D space in prior work, we propose a novel joint feature extractor to learn appearance and motion features from 2D and 3D space simultaneously. As features from different modalities often have complementary information, the joint feature can be more discriminate than feature from each individual modality. To ensure that the joint feature extractor does not heavily rely on one modality, we also propose an ensemble training paradigm. Through extensive evaluation, our proposed method achieves state-of-the-art performance on KITTI and nuScenes 3D MOT benchmarks. Our code will be made available at https://github.com/xinshuoweng/GNN3DMOT