mlsanityData to 14 Oct 2026

GRID: A Platform for General Robot Intelligence Development

arXiv 2310.00887 ↗First posted 2 Oct 2023cs.RO cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Developing machine intelligence abilities in robots and autonomous systems is an expensive and time consuming process. Existing solutions are tailored to specific applications and are harder to generalize. Furthermore, scarcity of training data adds a layer of complexity in deploying deep machine learning models. We present a new platform for General Robot Intelligence Development (GRID) to address both of these issues. The platform enables robots to learn, compose and adapt skills to their physical capabilities, environmental constraints and goals. The platform addresses AI problems in robotics via foundation models that know the physical world. GRID is designed from the ground up to be extensible to accommodate new types of robots, vehicles, hardware platforms and software protocols. In addition, the modular design enables various deep ML components and existing foundation models to be easily usable in a wider variety of robot-centric problems. We demonstrate the platform in various aerial robotics scenarios and demonstrate how the platform dramatically accelerates development of machine intelligent robots.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 108 of its 128 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

108 of 128 references.

References · 108

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  8. 2023Segment Anythingcited by 8,234
  9. 2023Visual Instruction Tuningcited by 7,869
  10. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  12. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
38 more
  1. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  9. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  11. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  12. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  13. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  14. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  15. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  16. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  17. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  18. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  19. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  20. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  21. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  22. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  23. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  24. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  25. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  26. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  27. 2023Large Language Models as Optimizerscited by 864
  28. 2022A Generalist Agentcited by 808
  29. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  30. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  31. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  32. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  33. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  34. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 537
  35. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  36. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  37. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  38. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511

50 most cited of 108 All 108 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4; 2025: 1.

6 papers.

  1. 2024Large Multimodal Agents: A Surveycited by 105
  2. 2024VoicePilot: Harnessing LLMs as Speech Interfaces for Physically Assistive Robotscited by 16
  3. 2023Agent as Cerebrum, Controller as Cerebellum: Implementing an Embodied LMM-based Agent on Dronescited by 9
  4. 2024On the Standardization of Behavioral Use Clauses and Their Adoption for Responsible Licensing of AIcited by 7
  5. 2024Towards an LLM-Based Speech Interface for Robot-Assisted Feedingcited by 2
  6. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2