# SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning

arXiv: https://arxiv.org/abs/2204.00734 · HTML: https://mlsanity.com/paper/2204.00734/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 63 references that are in the map.
- First posted on arXiv 2 Apr 2022.

## Builds on (most cited first, 30 of 36)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Ensemble Adversarial Training: Attacks and Defenses](https://mlsanity.com/paper/1705.07204/) (2017)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Adversarially Robust Generalization Requires More Data](https://mlsanity.com/paper/1804.11285/) (2018)
- [Parseval Networks: Improving Robustness to Adversarial Examples](https://mlsanity.com/paper/1704.08847/) (2017)
- [Adversarial Attacks and Defenses in Images, Graphs and Text: A Review](https://mlsanity.com/paper/1909.08072/) (2019)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark](https://mlsanity.com/paper/2103.16746/) (2021)
- [Deformable Siamese Attention Networks for Visual Object Tracking](https://mlsanity.com/paper/2004.06711/) (2020)
- [Adversarial Multi-task Learning for Text Classification](https://mlsanity.com/paper/1704.05742/) (2017)
- [Learning to Track at 100 FPS with Deep Regression Networks](https://mlsanity.com/paper/1604.01802/) (2016)
- [Unsupervised Traffic Accident Detection in First-Person Videos](https://mlsanity.com/paper/1903.00618/) (2019)
- [DepthTrack : Unveiling the Power of RGBD Tracking](https://mlsanity.com/paper/2108.13962/) (2021)
- [Efficient Adversarial Attacks for Visual Object Tracking](https://mlsanity.com/paper/2008.00217/) (2020)
- [Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty](https://mlsanity.com/paper/1711.09026/) (2017)
- [SiamMOT: Siamese Multi-Object Tracking](https://mlsanity.com/paper/2105.11595/) (2021)
- [Future Person Localization in First-Person Videos](https://mlsanity.com/paper/1711.11217/) (2017)
- [MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network](https://mlsanity.com/paper/1807.04067/) (2018)
- [Deep Defense: Training DNNs with Improved Adversarial Robustness](https://mlsanity.com/paper/1803.00404/) (2018)

## Built on by (most cited first, 2 of 2)

- [Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey](https://mlsanity.com/paper/2409.18169/) (2024)
- [RobArch: Designing Robust Architectures against Adversarial Attacks](https://mlsanity.com/paper/2301.03110/) (2023)

## Abstract (arXiv)

Person tracking using computer vision techniques has wide ranging applications such as autonomous driving, home security and sports analytics. However, the growing threat of adversarial attacks raises serious concerns regarding the security and reliability of such techniques. In this work, we study the impact of multi-task learning (MTL) on the adversarial robustness of the widely used SiamRPN tracker, in the context of person tracking. Specifically, we investigate the effect of jointly learning with semantically analogous tasks of person tracking and human keypoint detection. We conduct extensive experiments with more powerful adversarial attacks that can be physically realizable, demonstrating the practical value of our approach. Our empirical study with simulated as well as real-world datasets reveals that training with MTL consistently makes it harder to attack the SiamRPN tracker, compared to typically training only on the single task of person tracking.