mlsanityData to 7 Oct 2026

RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking

arXiv 2309.01918 ↗First posted 5 Sep 2023cs.RO cs.LGcited by 167

Not on the map. Why?

Cited by 44 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The grand aim of having a single robot that can manipulate arbitrary objects in diverse settings is at odds with the paucity of robotics datasets. Acquiring and growing such datasets is strenuous due to manual efforts, operational costs, and safety challenges. A path toward such an universal agent would require a structured framework capable of wide generalization but trained within a reasonable data budget. In this paper, we develop an efficient system (RoboAgent) for training universal agents capable of multi-task manipulation skills using (a) semantic augmentations that can rapidly multiply existing datasets and (b) action representations that can extract performant policies with small yet diverse multi-modal datasets without overfitting. In addition, reliable task conditioning and an expressive policy architecture enable our agent to exhibit a diverse repertoire of skills in novel situations specified using language commands. Using merely 7500 demonstrations, we are able to train a single agent capable of 12 unique skills, and demonstrate its generalization over 38 tasks spread across common daily activities in diverse kitchen scenes. On average, RoboAgent outperforms prior methods by over 40% in unseen situations while being more sample efficient and being amenable to capability improvements and extensions through fine-tuning. Videos at https://robopen.github.io/

Key facts. Cited by 167 papers in the map (data through 7 Oct 2026). Builds on 60 of its 72 references that are in the map. First posted on arXiv 5 Sep 2023.

Builds on

60 of 72 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2023Segment Anythingcited by 8,158
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  5. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  7. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  8. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  9. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
38 more
  1. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  2. 2022A Generalist Agentcited by 802
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  4. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  5. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  7. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  8. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  9. 2019Model-Based Reinforcement Learning for Ataricited by 495
  10. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  11. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  12. 2019Learning Latent Plans from Playcited by 310
  13. 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 303
  14. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  15. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  16. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  17. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  18. 2023Track Anything: Segment Anything Meets Videoscited by 230
  19. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  20. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  21. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
  22. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  23. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  24. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  25. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  26. 2022Human-to-Robot Imitation in the Wildcited by 157
  27. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  28. 2017Self-Supervised Visual Planning with Temporal Skip Connectionscited by 137
  29. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  30. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  31. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Roboticscited by 105
  32. 2023GenAug: Retargeting behaviors to unseen situations via Generative Augmentationcited by 103
  33. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  34. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  35. 2020RL-CycleGAN: Reinforcement Learning Aware Simulation-To-Realcited by 89
  36. 2021RRL: Resnet as representation for Reinforcement Learningcited by 89
  37. 2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
  38. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 70

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 50; 2025: 72; 2026: 33.

167 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  4. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  7. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  8. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  9. 2024TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulationcited by 331
  10. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  11. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
  12. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
38 more
  1. 2023FireAct: Toward Language Agent Fine-tuningcited by 247
  2. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  3. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  4. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 144
  5. 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
  6. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 142
  7. 2023Large Language Models for Robotics: A Surveycited by 132
  8. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  9. 2024Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulationcited by 116
  10. 2025GR-3 Technical Reportcited by 110
  11. 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
  12. 2024URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Imagescited by 106
  13. 2025FLARE: Robot Learning with Implicit World Modelingcited by 105
  14. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  15. 2023On Bringing Robots Homecited by 101
  16. 2025Reinforcement Learning with Action Chunkingcited by 97
  17. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  18. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  19. 2025CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsificationcited by 70
  20. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 66
  21. 2025Humanoid Policy ~ Human Policycited by 65
  22. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 63
  23. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 60
  24. 2023QUAR-VLA: Vision-Language-Action Model for Quadruped Robotscited by 55
  25. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 51
  26. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrationscited by 51
  27. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  28. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
  29. 2024Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Samplingcited by 46
  30. 2024Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learningcited by 46
  31. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  32. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  33. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  34. 2024VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulationcited by 43
  35. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  36. 2024Diffusion Transformer Policycited by 41
  37. 2025GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulationcited by 41
  38. 2023CoPAL: Corrective Planning of Robot Actions with Large Language Modelscited by 39

50 most cited of 167 All 167 (JSON)