# mmHRI: Towards Privacy-Preserving Human-Robot Interaction with Millimeter-Wave Radar

arXiv: https://arxiv.org/abs/2609.34220 · HTML: https://mlsanity.com/paper/2609.34220/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 21 of 21)

- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensing](https://mlsanity.com/paper/2305.10345/) (2023)
- [YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss](https://mlsanity.com/paper/2204.06806/) (2022)
- [mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar](https://mlsanity.com/paper/2209.05070/) (2022)
- [From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans](https://mlsanity.com/paper/2509.06607/) (2025)
- [Diffusion Model is a Good Pose Estimator from 3D RF-Vision](https://mlsanity.com/paper/2403.16198/) (2024)
- [milliFlow: Scene Flow Estimation on mmWave Radar Point Cloud for Human Motion Sensing](https://mlsanity.com/paper/2306.17010/) (2023)
- [A Survey of mmWave-based Human Sensing: Technology, Platform and Applications](https://mlsanity.com/paper/2308.03149/) (2023)
- [3DRIMR: 3D Reconstruction and Imaging via mmWave Radar based on Deep Learning](https://mlsanity.com/paper/2108.02858/) (2021)
- [RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark](https://mlsanity.com/paper/2407.13930/) (2024)
- [OmniVLA: Physically-Grounded Multimodal VLA with Unified Multi-Sensor Perception for Robotic Manipulation](https://mlsanity.com/paper/2511.01210/) (2025)
- [GazeVLA: Learning Human Intention for Robotic Manipulation](https://mlsanity.com/paper/2604.22615/) (2026)
- [Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulation](https://mlsanity.com/paper/2605.30282/) (2026)
- [Skeleton-based Action Recognition of People Handling Objects](https://mlsanity.com/paper/1901.06882/) (2019)
- [HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation](https://mlsanity.com/paper/2606.31682/) (2026)
- [WaveMan: mmWave-Based Room-Scale Human Interaction Perception for Humanoid Robots](https://mlsanity.com/paper/2601.07454/) (2026)
- [OpenRoboCare: A Multimodal Multi-Task Expert Demonstration Dataset for Robot Caregiving](https://mlsanity.com/paper/2511.13707/) (2025)
- [GIVE: Grounding Human Gestures in Vision-Language-Action Models](https://mlsanity.com/paper/2606.13435/) (2026)

## Built on by (most cited first, 1 of 1)

- [Learning to Assess Heartbeat Observability for mmWave Heart-Rate Sensing](https://mlsanity.com/paper/2610.03570/) (2026)

## Abstract (arXiv)

Assistive robots increasingly operate in many human-centered environments and perform various human-robot interaction (HRI) tasks, such as object delivery. However, most existing HRI systems rely on RGB cameras that continuously observe humans to respond to non-verbal commands, such as hand gestures. This raises privacy concerns in privacy- critical environments, such as hospital wards or restaurants, where direct camera observation of humans is restricted. To develop privacy-preserving HRI, we leverage millimeter-wave (mmWave) radar, which can sense human motion through privacy barriers without identifiable imagery. We propose mmHRI, the first multi-modal robot manipulation framework that achieves mmWave radar-guided privacy-preserving HRI. mmHRI introduces two key designs to mitigate the sparsity and temporal inconsistency of radar data in cluttered robot manipulation environments. First, we propose a dual-stream architecture that jointly learns from unfiltered raw radar tensors and radar point clouds to estimate both human actions and 3D poses. To mitigate signal inconsistency, mmHRI further incorporates a memory-based state-space model (MSSM) that retains historical radar features to reduce abrupt changes in pose/action. These estimated human states are then converted into structured textual robot instructions, which control a vision-language-action (VLA) policy for closed-loop robot manipulation and human-aware reactions. Our evaluation covers human action recognition and closed-loop delivery and retrieval. In the privacy-preserving curtain setting, mmHRI achieves 85.09% action-recognition accuracy, outperforming existing radar-based alternatives. Robot trials further demonstrate successful delivery and retrieval under visual occlusion, with stable task performance across unseen subjects, clutter configurations, and environments.