# Causal neural set filtering for online multi-target tracking

arXiv: https://arxiv.org/abs/2609.16054 · HTML: https://mlsanity.com/paper/2609.16054/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 13 Sep 2026.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors](https://mlsanity.com/paper/2211.09791/) (2022)
- [Global Tracking Transformers](https://mlsanity.com/paper/2203.13250/) (2022)
- [Generalized optimal sub-pattern assignment metric](https://mlsanity.com/paper/1601.05585/) (2016)
- [Poisson multi-Bernoulli mixture filter: direct derivation and implementation](https://mlsanity.com/paper/1703.04264/) (2017)
- [A metric on the space of finite sets of trajectories for evaluation of multi-target tracking algorithms](https://mlsanity.com/paper/1605.01177/) (2016)
- [Transformers for Multi-Object Tracking on Point Clouds](https://mlsanity.com/paper/2205.15730/) (2022)
- [Next Generation Multitarget Trackers: Random Finite Set Methods vs Transformer-based Deep Learning](https://mlsanity.com/paper/2104.00734/) (2021)
- [DeepDA: LSTM-based Deep Data Association Network for Multi-Targets Tracking in Clutter](https://mlsanity.com/paper/1907.09915/) (2019)
- [Probabilistic GOSPA: A Metric for Performance Evaluation of Multi-Object Filters with Uncertainties](https://mlsanity.com/paper/2412.11482/) (2024)
- [Learning a Neural Association Network for Self-supervised Multi-Object Tracking](https://mlsanity.com/paper/2411.11514/) (2024)

## Abstract (arXiv)

Transformer-based multi-target tracking (MTT) jointly learns data association and state estimation, but MT3/Track-MT3-style trackers repeatedly re-encode measurement windows, incurring redundant computation. We propose Causal Neural Set Filtering (CNSF)footnotehrefhttps://github.com/daihuangyu/CNSFCode: https://github.com/daihuangyu/CNSF, a neural set filter that encodes only current measurements while carrying past evidence in a structured recursive track state. CNSF combines exclusive Sinkhorn association, association-conditioned Kalman-shaped updates with moment matching, and recurrent Bernoulli lifecycle modeling with measurement-driven birth. These mechanisms impose soft one-to-one constraints, propagate association-induced state uncertainty, and support existence estimation under missed detections and birth--death transitions. On a held-out three-regime simulated test set, CNSF reduces mean GOSPA and T-GOSPA relative to Track-MT3 by 19.3% and 30.4%, with 55.9% fewer parameters and a 3.76× speedup in single-thread CPU inference.