mlsanityData to 14 Oct 2026

On Bringing Robots Home

arXiv 2311.16098 ↗First posted 27 Nov 2023cs.RO cs.AI cs.CVcited by 101

Not on the map. Why?

Cited by 26 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Throughout history, we have successfully integrated various machines into our homes. Dishwashers, laundry machines, stand mixers, and robot vacuums are a few recent examples. However, these machines excel at performing only a single task effectively. The concept of a "generalist machine" in homes - a domestic assistant that can adapt and learn from our needs, all while remaining cost-effective - has long been a goal in robotics that has been steadily pursued for decades. In this work, we initiate a large-scale effort towards this goal by introducing Dobb-E, an affordable yet versatile general-purpose system for learning robotic manipulation within household settings. Dobb-E can learn a new task with only five minutes of a user showing it how to do it, thanks to a demonstration collection tool ("The Stick") we built out of cheap parts and iPhones. We use the Stick to collect 13 hours of data in 22 homes of New York City, and train Home Pretrained Representations (HPR). Then, in a novel home environment, with five minutes of demonstrations and fifteen minutes of adapting the HPR model, we show that Dobb-E can reliably solve the task on the Stretch, a mobile robot readily available on the market. Across roughly 30 days of experimentation in homes of New York City and surrounding areas, we test our system in 10 homes, with a total of 109 tasks in different environments, and finally achieve a success rate of 81%. Beyond success percentages, our experiments reveal a plethora of unique challenges absent or ignored in lab robotics. These range from effects of strong shadows, to variable demonstration quality by non-expert users. With the hope of accelerating research on home robots, and eventually seeing robot butlers in every home, we open-source Dobb-E software stack and models, our data, and our hardware designs at https://dobb-e.com

Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). Builds on 73 of its 96 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

73 of 96 references.

References · 73

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  7. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  10. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  11. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
  12. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
38 more
  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  2. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  3. 2023LERF: Language Embedded Radiance Fieldscited by 543
  4. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  5. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  6. 2022Visual Language Maps for Robot Navigationcited by 365
  7. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  8. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  9. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  10. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  11. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  12. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  13. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  14. 2022Masked Visual Pre-training for Motor Controlcited by 248
  15. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  16. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  17. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  18. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 226
  19. 2022Interactive Language: Talking to Robots in Real Timecited by 218
  20. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  21. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  22. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 208
  23. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 204
  24. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 169
  25. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  26. 2023Language-Driven Representation Learning for Roboticscited by 165
  27. 2022Human-to-Robot Imitation in the Wildcited by 161
  28. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 154
  29. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 147
  30. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 144
  31. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 133
  32. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  33. 2020ACRONYM: A Large-Scale Grasp Dataset Based on Simulationcited by 117
  34. 2020Visual Imitation Made Easycited by 111
  35. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 107
  36. 2022Navigating to Objects in the Real Worldcited by 106
  37. 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
  38. 2018Jacquard: A Large Scale Dataset for Robotic Grasp Detectioncited by 87

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 37; 2025: 44; 2026: 19.

101 papers.

Sort:

By area:

  1. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  2. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  3. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  4. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  5. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 410
  6. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  7. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 328
  8. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  9. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  10. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
  11. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  12. 2024Behavior Generation with Latent Actionscited by 160
38 more
  1. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  2. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 146
  3. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  4. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 145
  5. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 134
  6. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 124
  7. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  8. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  9. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  10. 2024Vision Language Models are In-Context Value Learnerscited by 90
  11. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  12. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 75
  13. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 71
  14. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 66
  15. 2025WorldGym: World Model as An Environment for Policy Evaluationcited by 64
  16. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 64
  17. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 62
  18. 2025FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policiescited by 57
  19. 2025Training Strategies for Efficient Embodied Reasoningcited by 55
  20. 2026MolmoAct2: Action Reasoning Models for Real-world Deploymentcited by 50
  21. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 49
  22. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 44
  23. 2024DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Controlcited by 44
  24. 2025Emergence of Human to Robot Transfer in Vision-Language-Action Modelscited by 42
  25. 2024Diffusion Transformer Policycited by 41
  26. 2025CEED-VLA: Consistency Vision-Language-Action Model with Early-Exit Decodingcited by 38
  27. 2024VLM See, Robot Do: Human Demo Video to Robot Action Plan via Vision Language Modelcited by 37
  28. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learningcited by 35
  29. 2024OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Roboticscited by 34
  30. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  31. 2024ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedbackcited by 31
  32. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  33. 2025RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot Manipulationcited by 26
  34. 2024AnySkin: Plug-and-play Skin Sensing for Robotic Touchcited by 23
  35. 2025MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulationcited by 23
  36. 2024Continuously Improving Mobile Manipulation with Autonomous Real-World RLcited by 22
  37. 2024TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learningcited by 22
  38. 2024Learning Visual Quadrupedal Loco-Manipulation from Demonstrationscited by 20

50 most cited of 101 All 101 (JSON)