# Vision-based Perimeter Defense via Multiview Pose Estimation

arXiv: https://arxiv.org/abs/2209.12136 · HTML: https://mlsanity.com/paper/2209.12136/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 25 Sep 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [GDRNPP: A Geometry-guided and Fully Learning-based Object Pose Estimator](https://mlsanity.com/paper/2102.12145/) (2021)
- [Driving Policy Transfer via Modularity and Abstraction](https://mlsanity.com/paper/1804.09364/) (2018)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [Deep Drone Racing: From Simulation to Reality with Domain Randomization](https://mlsanity.com/paper/1905.09727/) (2019)
- [SLOAM: Semantic Lidar Odometry and Mapping for Forest Inventory](https://mlsanity.com/paper/1912.12726/) (2019)
- [Domain Randomization for Active Pose Estimation](https://mlsanity.com/paper/1903.03953/) (2019)
- [MAVNet: an Effective Semantic Segmentation Micro-Network for MAV-based Tasks](https://mlsanity.com/paper/1904.01795/) (2019)
- [Cooperative Team Strategies for Multi-player Perimeter-Defense Games](https://mlsanity.com/paper/1912.04342/) (2019)
- [Defending a Perimeter from a Ground Intruder Using an Aerial Defender: Theory and Practice](https://mlsanity.com/paper/2109.02852/) (2021)
- [Perimeter-defense Game between Aerial Defender and Ground Intruder](https://mlsanity.com/paper/2012.14980/) (2020)
- [Construction of the Barrier for Reach-Avoid Differential Games in Three-Dimensional Space with Four Equal-Speed Players](https://mlsanity.com/paper/1904.02810/) (2019)
- [MSL-RAPTOR: A 6DoF Relative Pose Tracker for Onboard Robotic Perception](https://mlsanity.com/paper/2012.09264/) (2020)

## Built on by (most cited first, 2 of 2)

- [Graph Neural Networks for Decentralized Multi-Agent Perimeter Defense](https://mlsanity.com/paper/2301.09689/) (2023)
- [Embedded Mean Field Reinforcement Learning for Perimeter-defense Game](https://mlsanity.com/paper/2505.14209/) (2025)

## Abstract (arXiv)

Previous studies in the perimeter defense game have largely focused on the fully observable setting where the true player states are known to all players. However, this is unrealistic for practical implementation since defenders may have to perceive the intruders and estimate their states. In this work, we study the perimeter defense game in a photo-realistic simulator and the real world, requiring defenders to estimate intruder states from vision. We train a deep machine learning-based system for intruder pose detection with domain randomization that aggregates multiple views to reduce state estimation errors and adapt the defensive strategy to account for this. We newly introduce performance metrics to evaluate the vision-based perimeter defense. Through extensive experiments, we show that our approach improves state estimation, and eventually, perimeter defense performance in both 1-defender-vs-1-intruder games, and 2-defenders-vs-1-intruder games.