# Syn2RealTrack: Bridging the Gap Between Synthetic and Real-World Datasets for Online Multi-View Multi-Target Tracking

arXiv: https://arxiv.org/abs/2608.24130 · HTML: https://mlsanity.com/paper/2608.24130/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 50 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Builds on (most cited first, 28 of 28)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [RF-DETR: Neural Architecture Search for Real-Time Detection Transformers](https://mlsanity.com/paper/2511.09554/) (2025)
- [Multiview Detection with Feature Perspective Transformation](https://mlsanity.com/paper/2007.07247/) (2020)
- [ViTPose++: Vision Transformer for Generic Body Pose Estimation](https://mlsanity.com/paper/2212.04246/) (2022)
- [Body Part-Based Representation Learning for Occluded Person Re-Identification](https://mlsanity.com/paper/2211.03679/) (2022)
- [GeoCalib: Learning Single-image Calibration with Geometric Optimization](https://mlsanity.com/paper/2409.06704/) (2024)
- [Multiview Detection with Shadow Transformer (and View-Coherent Data Augmentation)](https://mlsanity.com/paper/2108.05888/) (2021)
- [EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View](https://mlsanity.com/paper/2310.13350/) (2023)
- [Towards Domain Generalization for Multi-view 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2303.01686/) (2023)
- [AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration](https://mlsanity.com/paper/2503.12701/) (2025)
- [Wide-Baseline Multi-Camera Calibration using Person Re-Identification](https://mlsanity.com/paper/2104.08568/) (2021)
- [Keypoint Promptable Re-Identification](https://mlsanity.com/paper/2407.18112/) (2024)
- [Generalizable Multi-Camera 3D Pedestrian Detection](https://mlsanity.com/paper/2104.05813/) (2021)
- [Two-level Data Augmentation for Calibrated Multi-view Detection](https://mlsanity.com/paper/2210.10756/) (2022)
- [The 9th AI City Challenge](https://mlsanity.com/paper/2508.13564/) (2025)
- [MCBLT: Multi-Camera Multi-Object 3D Tracking in Long Videos](https://mlsanity.com/paper/2412.00692/) (2024)
- [The 10th AI City Challenge](https://mlsanity.com/paper/2608.17044/) (2026)
- [MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark](https://mlsanity.com/paper/2403.20225/) (2024)
- [DCHM: Depth-Consistent Human Modeling for Multiview Detection](https://mlsanity.com/paper/2507.14505/) (2025)
- [MVTrajecter: Multi-View Pedestrian Tracking with Trajectory Motion Cost and Trajectory Appearance Cost](https://mlsanity.com/paper/2509.01157/) (2025)
- [TSBOW -- Traffic Surveillance Benchmark for Occluded Vehicles Under Various Weather Conditions](https://mlsanity.com/paper/2602.05414/) (2026)

## Abstract (arXiv)

Multi-camera 3D perception systems for warehouse scenes are trained largely on synthetic data and evaluated on physically captured environments. The resulting synthetic-to-real gap, which corrupts ground-plane localization and cross-camera identity association, is usually treated as one deficiency for a single domain-adaptation module to absorb; we argue instead that it enters the pipeline at three separable points: the camera calibration, the object shape prior, and the assumption that the object census is known, each admitting a different local remedy. Our online pipeline, Syn2RealTrack, follows this decomposition: lens distortion is recovered from images alone under a calibration that provides none, detections are fused across views by a visibility-weighted part-based descriptor that abstains on occluded parts rather than guessing, person height is measured in closed form from calibration instead of copied from a synthetic prior, and a closed-world cardinality prior is paired with a causal filter that removes the phantom boxes the prior manufactures. The system therefore adapts by reallocating trust between geometry and appearance without retraining a feature extractor. On the AI City Challenge 2026 Track~1 evaluation server it reaches a 3D Higher Order Tracking Accuracy (HOTA) of 52.0118%. The code will be released at https://github.com/SKKUAutoLab/aic26_mc3dp