# FarSight: A Physics-Driven Whole-Body Biometric System at Large Distance and Altitude

arXiv: https://arxiv.org/abs/2306.17206 · HTML: https://mlsanity.com/paper/2306.17206/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 68 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 30 of 36)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [Harmonious Attention Network for Person Re-Identification](https://mlsanity.com/paper/1802.08122/) (2018)
- [DeepHuman: 3D Human Reconstruction from a Single Image](https://mlsanity.com/paper/1903.06473/) (2019)
- [Person Re-identification: Past, Present and Future](https://mlsanity.com/paper/1610.02984/) (2016)
- [DSFD: Dual Shot Face Detector](https://mlsanity.com/paper/1810.10220/) (2018)
- [Clothes-Changing Person Re-identification with RGB Modality Only](https://mlsanity.com/paper/2204.06890/) (2022)
- [OpenGait: Revisiting Gait Recognition Toward Better Practicality](https://mlsanity.com/paper/2211.06597/) (2022)
- [Deep Semantic Face Deblurring](https://mlsanity.com/paper/1803.03345/) (2018)
- [Cloth-Changing Person Re-identification from A Single Image with Gait Prediction and Regularization](https://mlsanity.com/paper/2103.15537/) (2021)
- [On the Reconstruction of Face Images from Deep Face Templates](https://mlsanity.com/paper/1703.00832/) (2017)
- [Multiple Object Tracking with Correlation Learning](https://mlsanity.com/paper/2104.03541/) (2021)
- [Expanding Accurate Person Recognition to New Altitudes and Ranges: The BRIAR Dataset](https://mlsanity.com/paper/2211.01917/) (2022)
- [Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform](https://mlsanity.com/paper/2107.11627/) (2021)
- [On Learning Disentangled Representations for Gait Recognition](https://mlsanity.com/paper/1909.03051/) (2019)
- [AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2208.11284/) (2022)
- [Towards Discriminative Representation: Multi-view Trajectory Contrastive Learning for Online Multi-object Tracking](https://mlsanity.com/paper/2203.14208/) (2022)
- [TinaFace: Strong but Simple Baseline for Face Detection](https://mlsanity.com/paper/2011.13183/) (2020)
- [Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition](https://mlsanity.com/paper/1803.11366/) (2018)
- [Simulating Anisoplanatic Turbulence by Sampling Inter-modal and Spatially Correlated Zernike Coefficients](https://mlsanity.com/paper/2004.11210/) (2020)
- [Controllable and Guided Face Synthesis for Unconstrained Face Recognition](https://mlsanity.com/paper/2207.10180/) (2022)
- [Cluster and Aggregate: Face Recognition with Large Probe Set](https://mlsanity.com/paper/2210.10864/) (2022)

## Built on by (most cited first, 27 of 27)

- [BigGait: Learning Gait Representation You Want by Large Vision Models](https://mlsanity.com/paper/2402.19122/) (2024)
- [KeyPoint Relative Position Encoding for Face Recognition](https://mlsanity.com/paper/2403.14852/) (2024)
- [Learning Clothing and Pose Invariant 3D Shape Representation for Long-Term Person Re-Identification](https://mlsanity.com/paper/2308.10658/) (2023)
- [Physics-Driven Turbulence Image Restoration with Stochastic Refinement](https://mlsanity.com/paper/2307.10603/) (2023)
- [BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models](https://mlsanity.com/paper/2505.18132/) (2025)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [SapiensID: Foundation for Human Recognition](https://mlsanity.com/paper/2504.04708/) (2025)
- [Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait](https://mlsanity.com/paper/2505.04616/) (2025)
- [Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data](https://mlsanity.com/paper/2412.01383/) (2024)
- [Open-Set Biometrics: Beyond Good Closed-Set Models](https://mlsanity.com/paper/2407.16133/) (2024)
- [A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognition](https://mlsanity.com/paper/2508.00053/) (2025)
- [HAMoBE: Hierarchical and Adaptive Mixture of Biometric Experts for Video-based Person ReID](https://mlsanity.com/paper/2508.05038/) (2025)
- [Whole-body Detection, Recognition and Identification at Altitude and Range](https://mlsanity.com/paper/2311.05725/) (2023)
- [From Data to Insights: A Covariate Analysis of the IARPA BRIAR Dataset for Multimodal Biometric Recognition Algorithms at Altitude and Range](https://mlsanity.com/paper/2409.01514/) (2024)
- [Generalizable Object Re-Identification via Visual In-Context Prompting](https://mlsanity.com/paper/2508.21222/) (2025)
- [LocalScore: Local Density-Aware Similarity Scoring for Biometrics](https://mlsanity.com/paper/2602.01012/) (2026)
- [FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognition](https://mlsanity.com/paper/2603.26908/) (2026)
- [Person Recognition in Aerial Surveillance: A Decade Survey](https://mlsanity.com/paper/2511.17674/) (2025)
- [VILLS -- Video-Image Learning to Learn Semantics for Person Re-Identification](https://mlsanity.com/paper/2311.17074/) (2023)
- [Source-free Domain Adaptation for Video Object Detection Under Adverse Image Conditions](https://mlsanity.com/paper/2404.15252/) (2024)
- [A Quantitative Evaluation of the Expressivity of BMI, Pose and Gender in Body Embeddings for Recognition and Identification](https://mlsanity.com/paper/2503.06451/) (2025)
- [IDSelect: A RL-Based Cost-Aware Selection Agent for Video-based Multi-Modal Person Recognition](https://mlsanity.com/paper/2602.18990/) (2026)
- [SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception](https://mlsanity.com/paper/2608.10497/) (2026)
- [AttriBE: Quantifying Attribute Expressivity in Body Embeddings for Recognition and Identification](https://mlsanity.com/paper/2604.27218/) (2026)
- [Subtoken Vision Transformer for Fine-grained Recognition](https://mlsanity.com/paper/2607.09086/) (2026)
- [Occluded Oculus: Operationalizing Stylistic Obscurement](https://mlsanity.com/paper/2607.24411/) (2026)
- [Restore What Matters: Lessons from Joint Restoration and Recognition](https://mlsanity.com/paper/2609.13791/) (2026)

## Abstract (arXiv)

Whole-body biometric recognition is an important area of research due to its vast applications in law enforcement, border security, and surveillance. This paper presents the end-to-end design, development and evaluation of FarSight, an innovative software system designed for whole-body (fusion of face, gait and body shape) biometric recognition. FarSight accepts videos from elevated platforms and drones as input and outputs a candidate list of identities from a gallery. The system is designed to address several challenges, including (i) low-quality imagery, (ii) large yaw and pitch angles, (iii) robust feature extraction to accommodate large intra-person variabilities and large inter-person similarities, and (iv) the large domain gap between training and test sets. FarSight combines the physics of imaging and deep learning models to enhance image restoration and biometric feature encoding. We test FarSight's effectiveness using the newly acquired IARPA Biometric Recognition and Identification at Altitude and Range (BRIAR) dataset. Notably, FarSight demonstrated a substantial performance increase on the BRIAR dataset, with gains of +11.82% Rank-20 identification and +11.3% TAR@1% FAR.