OceanGym: A Benchmark Environment for Underwater Embodied Agents
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
We introduce OceanGym, the first comprehensive benchmark for ocean underwater embodied agents, designed to advance AI in one of the most demanding real-world environments. Unlike terrestrial or aerial domains, underwater settings present extreme perceptual and decision-making challenges, including low visibility, dynamic ocean currents, making effective agent deployment exceptionally difficult. OceanGym encompasses eight realistic task domains and a unified agent framework driven by Multi-modal Large Language Models (MLLMs), which integrates perception, memory, and sequential decision-making. Agents are required to comprehend optical and sonar data, autonomously explore complex environments, and accomplish long-horizon objectives under these harsh conditions. Extensive experiments reveal substantial gaps between state-of-the-art MLLM-driven agents and human experts, highlighting the persistent difficulty of perception, planning, and adaptability in ocean underwater environments. By providing a high-fidelity, rigorously designed platform, OceanGym establishes a testbed for developing robust embodied AI and transferring these capabilities to real-world autonomous ocean underwater vehicles, marking a decisive step toward intelligent agents capable of operating in one of Earth's last unexplored frontiers. The code and data are available at https://github.com/OceanGPT/OceanGym.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 51 of its 83 references that are in the map. First posted on arXiv 30 Sep 2025.
Builds on
51 of 83 references.
Closest ancestors · 8
Same direction, newest first
- 2025Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence18 Jun
- 2025BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVs23 May
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
- 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents21 Jan
- 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment12 Oct
- 2024AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models28 Aug
- 2024OPEx: A Component-Wise Analysis of LLM-Centric Agents in Embodied Instruction Following5 Mar
- 2023An Embodied Generalist Agent in 3D World18 Nov
Other references · 43
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
31 more
- 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
- 2018VirtualHome: Simulating Household Activities via Programscited by 415
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
- 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
- 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
- 2021Embodied Intelligence via Learning and Evolutioncited by 163
- 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
- 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
- 2025Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solvingcited by 94
- 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
- 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learningcited by 80
- 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 79
- 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
- 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
- 2024An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Modelscited by 50
- 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 42
- 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 40
- 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36
- 2024HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environmentscited by 32
- 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigationcited by 24
- 2023MarineGPT: Unlocking Secrets of Ocean to the Publiccited by 23
- 2025NeuralOM: Neural Ocean Model for Subseasonal-to-Seasonal Simulationcited by 12
- 2025RynnEC: Bringing MLLMs into Embodied Worldcited by 11
- 2025OceanSim: A GPU-Accelerated Underwater Robot Perception Simulation Frameworkcited by 10
- 2025MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Roboticscited by 8
- 2024Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challengescited by 5
- 2025Underwater Robotic Simulators Review for Autonomous System Developmentcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026MobileMem: Learning from a Year of Mobile Experiencescited by 1
- 2026MobileMem: Learning from a Year of Mobile Experiences11 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Underwater Embodied Intelligence for Autonomous Robots: A Constraint-Coupled Perspective on Planning, Control, and Deployment ↗
- 2026AquaCap: A Training-Free Underwater Embodied Agent with Code-as-Policy ↗
- 2026AquaWAM: A Dynamics-aware World Action Model for Underwater Embodied Agents
- 2025UnderwaterVLA: Dual-brain Vision-Language-Action architecture for Autonomous Underwater Navigation
- 2026OceanMind: A multi-agent AI system for ocean diagnosis
- 2025Advancing Shared and Multi-Agent Autonomy in Underwater Missions: Integrating Knowledge Graphs and Retrieval-Augmented Generation
- 2026Assessing Vision-Language Models for Perception in Autonomous Underwater Robotic Software
- 2026CORAL: COntextual Reasoning And Local Planning in A Hierarchical VLM Framework for Underwater Monitoring