# PNAS-MOT: Multi-Modal Object Tracking with Pareto Neural Architecture Search

arXiv: https://arxiv.org/abs/2403.15712 · HTML: https://mlsanity.com/paper/2403.15712/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Builds on (most cited first, 30 of 31)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning](https://mlsanity.com/paper/2006.07327/) (2020)
- [FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detection](https://mlsanity.com/paper/2106.12449/) (2021)
- [EagerMOT: 3D Multi-Object Tracking via Sensor Fusion](https://mlsanity.com/paper/2104.14682/) (2021)
- [SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection](https://mlsanity.com/paper/2304.14340/) (2023)
- [Deep Affinity Network for Multiple Object Tracking](https://mlsanity.com/paper/1810.11780/) (2018)
- [Robust Multi-Modality Multi-Object Tracking](https://mlsanity.com/paper/1909.03850/) (2019)
- [Track to Reconstruct and Reconstruct to Track](https://mlsanity.com/paper/1910.00130/) (2019)
- [Deep Network Flow for Multi-Object Tracking](https://mlsanity.com/paper/1706.08482/) (2017)
- [Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking](https://mlsanity.com/paper/1802.09298/) (2018)
- [RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registration](https://mlsanity.com/paper/2303.12384/) (2023)
- [Mono-Camera 3D Multi-Object Tracking Using Deep Learning Detections and PMBM Filtering](https://mlsanity.com/paper/1802.09975/) (2018)
- [PolarMOT: How Far Can Geometric Relations Take Us in 3D Multi-Object Tracking?](https://mlsanity.com/paper/2208.01957/) (2022)
- [FANTrack: 3D Multi-Object Tracking with Feature Association Network](https://mlsanity.com/paper/1905.02843/) (2019)
- [Accurate Single Stage Detector Using Recurrent Rolling Convolution](https://mlsanity.com/paper/1704.05776/) (2017)
- [End-to-end Learning of Multi-sensor 3D Tracking by Detection](https://mlsanity.com/paper/1806.11534/) (2018)
- [HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search](https://mlsanity.com/paper/2102.11646/) (2021)
- [Exploiting More Information in Sparse Point Cloud for 3D Single Object Tracking](https://mlsanity.com/paper/2210.00519/) (2022)
- [DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds](https://mlsanity.com/paper/2308.04383/) (2023)
- [Weakly-Supervised Domain Adaptation of Deep Regression Trackers via Reinforced Knowledge Distillation](https://mlsanity.com/paper/2103.14496/) (2021)
- [FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation](https://mlsanity.com/paper/1407.6251/) (2014)
- [Vehicle Tracking Using Surveillance with Multimodal Data Fusion](https://mlsanity.com/paper/1811.02627/) (2018)

## Built on by (most cited first, 9 of 9)

- [DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes](https://mlsanity.com/paper/2411.11921/) (2024)
- [MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving](https://mlsanity.com/paper/2409.16149/) (2024)
- [Joint Pedestrian Trajectory Prediction through Posterior Sampling](https://mlsanity.com/paper/2404.00237/) (2024)
- [A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervision](https://mlsanity.com/paper/2412.00623/) (2024)
- [OmniTrack++: Omnidirectional Multi-Object Tracking by Learning Large-FoV Trajectory Feedback](https://mlsanity.com/paper/2511.00510/) (2025)
- [HybridTrack: A Hybrid Approach for Robust Multi-Object Tracking](https://mlsanity.com/paper/2501.01275/) (2025)
- [CrossTracker: Robust Multi-modal 3D Multi-Object Tracking via Cross Correction](https://mlsanity.com/paper/2411.18850/) (2024)
- [Polycepta: Object-Centric Appearance Estimation for Multi-Object Tracking](https://mlsanity.com/paper/2606.23604/) (2026)
- [RegTrack: Simplicity Beneath Complexity in Robust Multi-Modal 3D Multi-Object Tracking](https://mlsanity.com/paper/2409.00618/) (2024)

## Abstract (arXiv)

Multiple object tracking is a critical task in autonomous driving. Existing works primarily focus on the heuristic design of neural networks to obtain high accuracy. As tracking accuracy improves, however, neural networks become increasingly complex, posing challenges for their practical application in real driving scenarios due to the high level of latency. In this paper, we explore the use of the neural architecture search (NAS) methods to search for efficient architectures for tracking, aiming for low real-time latency while maintaining relatively high accuracy. Another challenge for object tracking is the unreliability of a single sensor, therefore, we propose a multi-modal framework to improve the robustness. Experiments demonstrate that our algorithm can run on edge devices within lower latency constraints, thus greatly reducing the computational requirements for multi-modal object tracking while keeping lower latency.