mlsanityData to 14 Oct 2026

OceanGym: A Benchmark Environment for Underwater Embodied Agents

arXiv 2509.26536 ↗First posted 30 Sep 2025cs.CL cs.AI cs.CVcited by 1

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

We introduce OceanGym, the first comprehensive benchmark for ocean underwater embodied agents, designed to advance AI in one of the most demanding real-world environments. Unlike terrestrial or aerial domains, underwater settings present extreme perceptual and decision-making challenges, including low visibility, dynamic ocean currents, making effective agent deployment exceptionally difficult. OceanGym encompasses eight realistic task domains and a unified agent framework driven by Multi-modal Large Language Models (MLLMs), which integrates perception, memory, and sequential decision-making. Agents are required to comprehend optical and sonar data, autonomously explore complex environments, and accomplish long-horizon objectives under these harsh conditions. Extensive experiments reveal substantial gaps between state-of-the-art MLLM-driven agents and human experts, highlighting the persistent difficulty of perception, planning, and adaptability in ocean underwater environments. By providing a high-fidelity, rigorously designed platform, OceanGym establishes a testbed for developing robust embodied AI and transferring these capabilities to real-world autonomous ocean underwater vehicles, marking a decisive step toward intelligent agents capable of operating in one of Earth's last unexplored frontiers. The code and data are available at https://github.com/OceanGPT/OceanGym.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 51 of its 83 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

51 of 83 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence18 Jun
  2. 2025BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVs23 May
  3. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
  4. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents21 Jan
  5. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment12 Oct
  6. 2024AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models28 Aug
  7. 2024OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Following5 Mar
  8. 2023An Embodied Generalist Agent in 3D World18 Nov

Other references · 43

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  8. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  9. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  10. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  11. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  12. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
31 more
  1. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  2. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  3. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  4. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  5. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  6. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  7. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
  8. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  9. 2021Embodied Intelligence via Learning and Evolutioncited by 163
  10. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  11. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  12. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  13. 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 94
  14. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  15. 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learningcited by 80
  16. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 79
  17. 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
  18. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
  19. 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Modelscited by 50
  20. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 42
  21. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 40
  22. 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36
  23. 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32
  24. 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigationcited by 24
  25. 2023MarineGPT: Unlocking Secrets of Ocean to the Publiccited by 23
  26. 2025NeuralOM: Neural Ocean Model for Subseasonal-to-Seasonal Simulationcited by 12
  27. 2025RynnEC: Bringing MLLMs into Embodied Worldcited by 11
  28. 2025OceanSim: A GPU-Accelerated Underwater Robot Perception Simulation Frameworkcited by 10
  29. 2025MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Roboticscited by 8
  30. 2024Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challengescited by 5
  31. 2025Underwater Robotic Simulators Review for Autonomous System Developmentcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Underwater Embodied Intelligence for Autonomous Robots: A Constraint-Coupled Perspective on Planning, Control, and Deployment ↗
  2. 2026AquaCap: A Training-Free Underwater Embodied Agent with Code-as-Policy ↗
  3. 2026AquaWAM: A Dynamics-aware World Action Model for Underwater Embodied Agents
  4. 2025UnderwaterVLA: Dual-brain Vision-Language-Action architecture for Autonomous Underwater Navigation
  5. 2026OceanMind: A multi-agent AI system for ocean diagnosis
  6. 2025Advancing Shared and Multi-Agent Autonomy in Underwater Missions: Integrating Knowledge Graphs and Retrieval-Augmented Generation
  7. 2026Assessing Vision-Language Models for Perception in Autonomous Underwater Robotic Software
  8. 2026CORAL: COntextual Reasoning And Local Planning in A Hierarchical VLM Framework for Underwater Monitoring