mlsanityData to 7 Oct 2026

Towards Distraction-Robust Active Visual Tracking

arXiv 2106.10110 ↗First posted 18 Jun 2021cs.CV cs.AI cs.MAcited by 30

Not on the map. Why?

Abstract (arXiv)

In active visual tracking, it is notoriously difficult when distracting objects appear, as distractors often mislead the tracker by occluding the target or bringing a confusing appearance. To address this issue, we propose a mixed cooperative-competitive multi-agent game, where a target and multiple distractors form a collaborative team to play against a tracker and make it fail to follow. Through learning in our game, diverse distracting behaviors of the distractors naturally emerge, thereby exposing the tracker's weakness, which helps enhance the distraction-robustness of the tracker. For effective learning, we then present a bunch of practical methods, including a reward function for distractors, a cross-modal teacher-student learning strategy, and a recurrent attention mechanism for the tracker. The experimental results show that our tracker performs desired distraction-robust active visual tracking and can be well generalized to unseen environments. We also show that the multi-agent game can be used to adversarially test the robustness of trackers.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 29 of its 48 references that are in the map. First posted on arXiv 18 Jun 2021.

Builds on

29 of 48 references.

References · 29

  1. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  3. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  4. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  5. 2018Learning Dexterous In-Hand Manipulationcited by 643
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  7. 2017Robust Adversarial Reinforcement Learningcited by 439
  8. 2019Learning Discriminative Model Prediction for Trackingcited by 344
  9. 2017Adversarial Attacks on Neural Network Policiescited by 314
  10. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
  11. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  12. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
17 more
  1. 2015Learning Multi-Domain Convolutional Neural Networks for Visual Trackingcited by 209
  2. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
  3. 2018Distractor-aware Siamese Networks for Visual Object Trackingcited by 187
  4. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 180
  5. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  6. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  7. 2015A Novel Performance Evaluation Methodology for Single-Target Trackerscited by 113
  8. 2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
  9. 2018Multi-Agent Generative Adversarial Imitation Learningcited by 99
  10. 2017Coordinated Multi-Agent Imitation Learningcited by 70
  11. 2015Deep Reinforcement Learning in Parameterized Action Spacecited by 65
  12. 2018End-to-end Active Object Tracking and Its Real-world Deployment via Reinforcement Learningcited by 35
  13. 2017End-to-end Active Object Tracking via Reinforcement Learningcited by 27
  14. 2020Learning Multi-Agent Coordination for Enhancing Target Coverage in Directional Sensor Networkscited by 19
  15. 2016Inverse Reinforcement Learning in Swarm Systemscited by 18
  16. 2020Pose-Assisted Multi-Camera Collaboration for Active Object Trackingcited by 16
  17. 2019Learning to Manipulate Object Collections Using Grounded State Representationscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 3; 2024: 7; 2025: 8; 2026: 4.

30 papers.

Sort:

By area:

  1. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  2. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54
  3. 2025TrackVLA: Embodied Visual Tracking in the Wildcited by 46
  4. 2021Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Soundscited by 40
  5. 2024Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacycited by 36
  6. 2024UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AIcited by 25
  7. 2024Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigationcited by 24
  8. 2024Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RLcited by 21
  9. 2024Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Modelcited by 19
  10. 2023RSPT: Reconstruct Surroundings and Predict Trajectories for Generalizable Active Object Trackingcited by 16
  11. 2025TrackVLA++: Unleashing Reasoning and Memory Capabilities in VLA Models for Embodied Visual Trackingcited by 16
  12. 2022TarGF: Learning Target Gradient Field to Rearrange Objects without Explicit Goal Specificationcited by 13
18 more
  1. 2023Proactive Multi-Camera Collaboration For 3D Human Pose Estimationcited by 13
  2. 2022Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RLcited by 11
  3. 2025LOVON: Legged Open-Vocabulary Object Navigatorcited by 11
  4. 2024Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllerscited by 9
  5. 2022Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Trackingcited by 7
  6. 2025VLM Can Be a Good Assistant: Enhancing Embodied Visual Tracking with Self-Improving Vision-Language Modelscited by 7
  7. 2025Distracted Robot: How Visual Clutter Undermine Robotic Manipulationcited by 5
  8. 2026CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Modelscited by 5
  9. 2022Beyond Visual Field of View: Perceiving 3D Environment with Echoes and Visioncited by 4
  10. 2025Hierarchical Instruction-aware Embodied Visual Trackingcited by 4
  11. 2022Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigationcited by 3
  12. 2025Improving Robotic Manipulation Robustness via NICE Scene Surgerycited by 3
  13. 2026Instance-level Visual Active Tracking with Occlusion-Aware Planningcited by 3
  14. 2024SocialGFs: Learning Social Gradient Fields for Multi-Agent Reinforcement Learningcited by 2
  15. 2025EASE: Embodied Active Event Perception via Self-Supervised Energy Minimizationcited by 2
  16. 2022Dynamical Audio-Visual Navigation: Catching Unheard Moving Sound Sources in Unmapped 3D Environmentscited by 0
  17. 2026Task-Relevant Representation Decoupling for Visual Reinforcement Learning Generalizationcited by 0
  18. 2026CST-WM: A Causally Structured World Model for Embodied Visual Trackingcited by 0