mlsanityData to 7 Oct 2026

GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation

arXiv 2410.06158 ↗First posted 8 Oct 2024cs.RO cs.CV cs.LGcited by 290

Not on the map. Why?

Cited by 121 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present GR-2, a state-of-the-art generalist robot agent for versatile and generalizable robot manipulation. GR-2 is first pre-trained on a vast number of Internet videos to capture the dynamics of the world. This large-scale pre-training, involving 38 million video clips and over 50 billion tokens, equips GR-2 with the ability to generalize across a wide range of robotic tasks and environments during subsequent policy learning. Following this, GR-2 is fine-tuned for both video generation and action prediction using robot trajectories. It exhibits impressive multi-task learning capabilities, achieving an average success rate of 97.7% across more than 100 tasks. Moreover, GR-2 demonstrates exceptional generalization to new, previously unseen scenarios, including novel backgrounds, environments, objects, and tasks. Notably, GR-2 scales effectively with model size, underscoring its potential for continued growth and application. Project page: urlhttps://gr2-manipulation.github.io.

Key facts. Cited by 290 papers in the map (data through 7 Oct 2026). Builds on 63 of its 70 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

63 of 70 references.

References · 63

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2023Segment Anythingcited by 8,158
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  9. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  10. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  11. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  12. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
38 more
  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2018World Modelscited by 1,444
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  4. 2023Mastering Diverse Domains through World Modelscited by 1,193
  5. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  6. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  7. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  8. 2022A Generalist Agentcited by 802
  9. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  11. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  12. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  13. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  14. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  15. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  16. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  17. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
  18. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  19. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  20. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  21. 2023Learning Interactive Real-World Simulatorscited by 440
  22. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  23. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  24. 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
  25. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  26. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
  27. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  28. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  29. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  30. 2022Masked Visual Pre-training for Motor Controlcited by 246
  31. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  32. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  33. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  34. 2022Masked World Models for Visual Controlcited by 169
  35. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  36. 2023Language-Driven Representation Learning for Roboticscited by 163
  37. 2023Video Language Planningcited by 160
  38. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 127; 2026: 152.

290 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  3. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  4. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  5. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  6. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  7. 2026World Action Models are Zero-shot Policiescited by 411
  8. 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
  9. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
  10. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  11. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  12. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
38 more
  1. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  2. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  3. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 140
  4. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  5. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
  6. 2025Unified Vision-Language-Action Modelcited by 128
  7. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  8. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  9. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  10. 2025GR-3 Technical Reportcited by 110
  11. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 106
  12. 2025FLARE: Robot Learning with Implicit World Modelingcited by 105
  13. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 102
  14. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  15. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  16. 2024What Matters in Building Vision-Language-Action Models for Generalist Robotscited by 88
  17. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  18. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  19. 2025Universal Actions for Enhanced Embodied Foundation Modelscited by 80
  20. 2025BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Modelscited by 77
  21. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 74
  22. 2025Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policycited by 73
  23. 2025CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsificationcited by 70
  24. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68
  25. 2025VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizerscited by 67
  26. 2024Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videoscited by 65
  27. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  28. 2025Training-Time Action Conditioning for Efficient Real-Time Chunkingcited by 61
  29. 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulatorscited by 61
  30. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 61
  31. 2025EnerVerse: Envisioning Embodied Future Space for Robotics Manipulationcited by 59
  32. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 56
  33. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  34. 2025TLA: Tactile-Language-Action Model for Contact-Rich Manipulationcited by 46
  35. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  36. 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 46
  37. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  38. 2025Humanoid-VLA: Towards Universal Humanoid Control with Visual Integrationcited by 43

50 most cited of 290 All 290 (JSON)