mlsanityData to 7 Oct 2026

OmniRobotHome: A Multi-Camera Home Platform for Real-Time Human-Robot Interaction

arXiv 2604.28197 ↗First posted 30 Apr 2026cs.RO cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Robots in homes must continuously sense the people around them, yet most prior work relies on limited or offline perception. We argue that perception quality is the dominant factor governing what interaction is achievable at home, and build a testbed to test this claim. OmniRobotHome instruments a furnished home with 48 hardware-synchronized cameras and three manipulators in a unified world frame, delivering real-time markerless full-body human pose, 6D object pose, anticipatory motion forecasting, and a social avatar agent that converses with residents. Using the platform, we treat perception quality as an experimental variable across safety, human assistance, and social interaction, and find that interaction quality degrades measurably as real-timeness, granularity, coverage, accuracy, forecasting, or memory is weakened. All code and data will be released.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 80 references that are in the map. First posted on arXiv 30 Apr 2026.

Builds on

51 of 80 references.

References · 51

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  4. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  7. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  9. 2025OpenAI GPT-5 System Cardcited by 910
  10. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
38 more
  1. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  2. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  3. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 411
  4. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  5. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  6. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  7. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  8. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  9. 2023GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussianscited by 256
  10. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  11. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  12. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  13. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  14. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  15. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 170
  16. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  17. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  18. 2016Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypothesescited by 96
  19. 2019Fast and Robust Multi-Person 3D Pose Estimation from Multiple Viewscited by 83
  20. 2025YOLO26: Key Architectural Enhancements and Performance Benchmarking for Real-Time Object Detectioncited by 76
  21. 2021SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimationcited by 68
  22. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 54
  23. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 52
  24. 2021STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulationcited by 46
  25. 2018Riemannian Motion Policiescited by 45
  26. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 30
  27. 2025ARTalk: Speech-Driven 3D Head Animation via Autoregressive Modelcited by 28
  28. 2025Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matchingcited by 23
  29. 2024BODex: Scalable and Efficient Robotic Dexterous Grasp Synthesis Using Bilevel Optimizationcited by 18
  30. 2021Co-GAIL: Learning Diverse Strategies for Human-Robot Collaborationcited by 16
  31. 2026MV-SAM3D: Adaptive Multi-View Fusion for Layout-Aware 3D Generationcited by 16
  32. 2023InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actionscited by 10
  33. 2024MOSAIC: Modular Foundation Models for Assistive and Interactive Cookingcited by 9
  34. 2025FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalizationcited by 7
  35. 2025Deep Reactive Policy: Learning Reactive Manipulator Motion Planning for Dynamic Environmentscited by 7
  36. 2024ContactHandover: Contact-Guided Robot-to-Human Object Handovercited by 3
  37. 2025Context-aware collaborative pushing of heavy objects using skeleton-based intention predictioncited by 3
  38. 2017A Nonparametric Motion Flow Model for Human Robot Cooperationcited by 2

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.