mlsanityData to 7 Oct 2026

Harmonic Mobile Manipulation

arXiv 2312.06639 ↗First posted 11 Dec 2023cs.RO cs.AI cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

Recent advancements in robotics have enabled robots to navigate complex scenes or manipulate diverse objects independently. However, robots are still impotent in many household tasks requiring coordinated behaviors such as opening doors. The factorization of navigation and manipulation, while effective for some tasks, fails in scenarios requiring coordinated actions. To address this challenge, we introduce, HarmonicMM, an end-to-end learning method that optimizes both navigation and manipulation, showing notable improvement over existing techniques in everyday tasks. This approach is validated in simulated and real-world environments and adapts to novel unseen settings without additional tuning. Our contributions include a new benchmark for mobile manipulation and the successful deployment with only RGB visual observation in a real unseen apartment, demonstrating the potential for practical indoor robot deployment in daily life. More results are on our project site: https://rchalyang.github.io/HarmonicMM/

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 46 of its 58 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

46 of 58 references.

References · 46

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  7. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  9. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  10. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  11. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  12. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
34 more
  1. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  2. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  3. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
  4. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  5. 2023Large Language Models as General Pattern Machinescited by 196
  6. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  7. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  8. 2022Human-to-Robot Imitation in the Wildcited by 157
  9. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 149
  10. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 147
  11. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 131
  12. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  13. 2022Navigating to Objects in the Real Worldcited by 104
  14. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  15. 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 101
  16. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 98
  17. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 78
  18. 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
  19. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 60
  20. 2020AllenAct: A Framework for Embodied AI Researchcited by 54
  21. 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 52
  22. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 47
  23. 2022Multi-skill Mobile Manipulation for Object Rearrangementcited by 46
  24. 2023Neural Volumetric Memory for Visual Locomotion Controlcited by 40
  25. 2021The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human Environmentscited by 39
  26. 2023ASC: Adaptive Skill Coordination for Robotic Mobile Manipulationcited by 38
  27. 2021Fully Autonomous Real-World Reinforcement Learning with Applications to Mobile Manipulationcited by 30
  28. 2019DoorGym: A Scalable Door Opening Environment And Baseline Agentcited by 26
  29. 2022Phone2Proc: Bringing Robust Robots Into Our Chaotic Worldcited by 24
  30. 2022Learning Visual Locomotion with Cross-Modal Supervisioncited by 23
  31. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 23
  32. 2022Robot Learning of Mobile Manipulation with Reachability Behavior Priorscited by 20
  33. 2023Causal Policy Gradient for Whole-Body Mobile Manipulationcited by 18
  34. 2022Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimizationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 4; 2026: 6.

16 papers.

Sort:

By area:

  1. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  2. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
  3. 2025MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulationcited by 34
  4. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 30
  5. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  6. 2024TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learningcited by 21
  7. 2025AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulationcited by 21
  8. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 20
  9. 2026Learning Panorama-Aware VLA for Mobile Manipulation with Whole-Body Teleoperationcited by 4
  10. 2025Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Reviewcited by 3
  11. 2024Opening Articulated Structures in the Real Worldcited by 1
  12. 2026MoPA: Coordinated Mobile Manipulation via Subsystem-Specific Perception Alignmentcited by 1
4 more
  1. 2026EgoActor: Grounding Task Planning into Spatial-aware Egocentric Actions for Humanoid Robots via Visual-Language Modelscited by 0
  2. 2026YOR: Your Own Mobile Manipulator for Generalizable Roboticscited by 0
  3. 2026Whole-Body Mobile Manipulation using Offline Reinforcement Learning on Sub-optimal Controllerscited by 0
  4. 2026SEVO: Semantic-Enhanced Virtual Observation for Robust VLA Manipulation via Active Illumination and Data-Centric Collectioncited by 0