# Deep Fusion of Multi-Object Densities Using Transformer

arXiv: https://arxiv.org/abs/2209.08857 · HTML: https://mlsanity.com/paper/2209.08857/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [Generalized optimal sub-pattern assignment metric](https://mlsanity.com/paper/1601.05585/) (2016)
- [Fusion of Probability Density Functions](https://mlsanity.com/paper/2202.11633/) (2022)
- [A metric on the space of finite sets of trajectories for evaluation of multi-target tracking algorithms](https://mlsanity.com/paper/1605.01177/) (2016)
- [Multiple target tracking based on sets of trajectories](https://mlsanity.com/paper/1605.08163/) (2016)
- [Trajectory Poisson multi-Bernoulli filters](https://mlsanity.com/paper/2003.12767/) (2020)
- [Poisson multi-Bernoulli mixture trackers: continuity through random finite sets of trajectories](https://mlsanity.com/paper/1812.05131/) (2018)
- [An Uncertainty-Aware Performance Measure for Multi-Object Tracking](https://mlsanity.com/paper/2108.04619/) (2021)
- [Next Generation Multitarget Trackers: Random Finite Set Methods vs Transformer-based Deep Learning](https://mlsanity.com/paper/2104.00734/) (2021)
- [Can Deep Learning be Applied to Model-Based Multi-Object Tracking?](https://mlsanity.com/paper/2202.07909/) (2022)
- [Decentralized Poisson Multi-Bernoulli Filtering for Vehicle Tracking](https://mlsanity.com/paper/1901.04518/) (2019)
- [Heterogeneous Multi-sensor Fusion with Random Finite Set Multi-object Densities](https://mlsanity.com/paper/2106.08088/) (2021)
- [Fusion of labeled RFS densities with minimum information loss](https://mlsanity.com/paper/1911.01083/) (2019)
- [Fusion of finite set distributions: Pointwise consistency and global cardinality](https://mlsanity.com/paper/1802.06220/) (2018)

## Abstract (arXiv)

In this paper, we demonstrate that deep learning based method can be used to fuse multi-object densities. Given a scenario with several sensors with possibly different field-of-views, tracking is performed locally in each sensor by a tracker, which produces random finite set multi-object densities. To fuse outputs from different trackers, we adapt a recently proposed transformer-based multi-object tracker, where the fusion result is a global multi-object density, describing the set of all alive objects at the current time. We compare the performance of the transformer-based fusion method with a well-performing model-based Bayesian fusion method in several simulated scenarios with different parameter settings using synthetic data. The simulation results show that the transformer-based fusion method outperforms the model-based Bayesian method in our experimental scenarios.