# Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars

arXiv: https://arxiv.org/abs/2207.07896 · HTML: https://mlsanity.com/paper/2207.07896/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 49 references that are in the map.
- First posted on arXiv 16 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Harmonious Attention Network for Person Re-Identification](https://mlsanity.com/paper/1802.08122/) (2018)
- [Deep Metric Learning with Hierarchical Triplet Loss](https://mlsanity.com/paper/1810.06951/) (2018)
- [Unsupervised Person Re-identification via Multi-label Classification](https://mlsanity.com/paper/2004.09228/) (2020)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar](https://mlsanity.com/paper/1911.00398/) (2019)
- [Robust Depth-based Person Re-identification](https://mlsanity.com/paper/1703.09474/) (2017)
- [Recurrent Attention Models for Depth-Based Person Identification](https://mlsanity.com/paper/1611.07212/) (2016)
- [Recover and Identify: A Generative Dual Model for Cross-Resolution Person Re-Identification](https://mlsanity.com/paper/1908.06052/) (2019)
- [Multi-Person Continuous Tracking and Identification from mm-Wave micro-Doppler Signatures](https://mlsanity.com/paper/2003.03571/) (2020)
- [Adversarial Multi-scale Feature Learning for Person Re-identification](https://mlsanity.com/paper/2012.14061/) (2020)

## Built on by (most cited first, 8 of 8)

- [milliFlow: Scene Flow Estimation on mmWave Radar Point Cloud for Human Motion Sensing](https://mlsanity.com/paper/2306.17010/) (2023)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [Large Model for Small Data: Foundation Model for Cross-Modal RF Human Activity Recognition](https://mlsanity.com/paper/2410.19766/) (2024)
- [Non-contact Multimodal Indoor Human Monitoring Systems: A Survey](https://mlsanity.com/paper/2312.07601/) (2023)
- [ESP-PCT: Enhanced VR Semantic Performance through Efficient Compression of Temporal and Spatial Redundancies in Point Cloud Transformers](https://mlsanity.com/paper/2409.01216/) (2024)
- [Human Semantic Segmentation using Millimeter-Wave Radar Sparse Point Clouds](https://mlsanity.com/paper/2304.14132/) (2023)
- [mmSpyVR: Exploiting mmWave Radar for Penetrating Obstacles to Uncover Privacy Vulnerability of Virtual Reality](https://mlsanity.com/paper/2411.09914/) (2024)
- [Exploring Spatial-Temporal Representation via Star Graph for mmWave Radar-based Human Activity Recognition](https://mlsanity.com/paper/2512.12013/) (2025)

## Abstract (arXiv)

Human identification is a key requirement for many applications in everyday life, such as personalized services, automatic surveillance, continuous authentication, and contact tracing during pandemics, etc. This work studies the problem of cross-modal human re-identification (ReID), in response to the regular human movements across camera-allowed regions (e.g., streets) and camera-restricted regions (e.g., offices) deployed with heterogeneous sensors. By leveraging the emerging low-cost RGB-D cameras and mmWave radars, we propose the first-of-its-kind vision-RF system for cross-modal multi-person ReID at the same time. Firstly, to address the fundamental inter-modality discrepancy, we propose a novel signature synthesis algorithm based on the observed specular reflection model of a human body. Secondly, an effective cross-modal deep metric learning model is introduced to deal with interference caused by unsynchronized data across radars and cameras. Through extensive experiments in both indoor and outdoor environments, we demonstrate that our proposed system is able to achieve ~92.5% top-1 accuracy and ~97.5% top-5 accuracy out of 56 volunteers. We also show that our proposed system is able to robustly reidentify subjects even when multiple subjects are present in the sensors' field of view.