mlsanityData to 7 Oct 2026

SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulation

arXiv 2608.04196 ↗First posted 4 Aug 2026cs.RO cs.CV cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed an explosive trend of scaling ego-centric human videos for robot manipulation, yet it remains unclear which data actually benefits dexterous manipulation. We present SiMDex, a similarity-based data mining framework that casts human data selection for VLA post-training in dexterous manipulation as a recommendation problem. For each robot demonstration, SiMDex employs a three-layer recall-ranking-re-ranking pipeline to extract task-relevant subsets from a pool of ~32M egocentric human samples, operating in a morphology-agnostic action space that requires no changes to VLA architecture or training. Against a strong baseline trained with an equal amount of randomly sampled human data, SiMDex uses only ~1.49M mined samples (<5% of the pool) yet improves the overall success rate from 47.7% to 61.1%, showing that selective curation outperforms indiscriminate data mixing.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 50 of its 52 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

50 of 52 references.

References · 50

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  6. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  8. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  9. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  10. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  11. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  12. 2020Rescaling Egocentric Visioncited by 529
38 more
  1. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  2. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  3. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  4. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 295
  5. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  6. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  7. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
  8. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  9. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
  10. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 160
  11. 2022Human-to-Robot Imitation in the Wildcited by 157
  12. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  13. 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversificationcited by 146
  14. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 116
  15. 2025GR-3 Technical Reportcited by 110
  16. 2019DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm Systemcited by 105
  17. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  18. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  19. 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Datacited by 77
  20. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  21. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 56
  22. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 49
  23. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learningcited by 44
  24. 2025Masquerade: Learning from In-the-wild Human Videos using Data-Editingcited by 38
  25. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 36
  26. 2025EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Datacited by 27
  27. 2025Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skillscited by 21
  28. 2025CUPID: Curating Data your Robot Loves with Influence Functionscited by 21
  29. 2026EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Taskscited by 15
  30. 2026DexImit: Learning Bimanual Dexterous Manipulation from Monocular Human Videoscited by 11
  31. 2025Dexterous Teleoperation of 20-DoF ByteDexter Hand via Human Motion Retargetingcited by 9
  32. 2026HumanNet: Scaling Human-centric Video Learning to One Million Hourscited by 9
  33. 2025SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-trainingcited by 7
  34. 2025GR-Dexter Technical Reportcited by 4
  35. 2025EgoBrain: Synergizing Minds and Eyes For Human Action Understandingcited by 2
  36. 2025Let Me Show You: Learning by Retrieving from Egocentric Video for Robotic Manipulationcited by 2
  37. 2026YUBI: Yielding Universal Bidigital Interface for Bimanual Dexterous Manipulation at Scalecited by 2
  38. 2026Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robotscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learningcited by 0
  2. 2026ReWeight: Leveraging Human Data for VLA Post-Training via Demonstration Retrieval and Sample Weightingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Data
  2. 2025OpenEgo: A Large-Scale Multimodal Egocentric Dataset for Dexterous Manipulation
  3. 2026Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data
  4. 2026Wh0: Generative World Models as Scalable Sources of Egocentric Human Hand Manipulation Data
  5. 2026IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining ↗
  6. 2026EgoSteer: An Open-Source Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos
  7. 2026Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning
  8. 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models