# Gravity-Bench-v1: A Benchmark on Gravitational Physics Discovery for Agents

arXiv: https://arxiv.org/abs/2501.18411 · HTML: https://mlsanity.com/paper/2501.18411/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 30 Jan 2025.

## Builds on (most cited first, 14 of 14)

- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://mlsanity.com/paper/2404.07738/) (2024)
- [LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery](https://mlsanity.com/paper/2405.09783/) (2024)
- [LLM4SR: A Survey on Large Language Models for Scientific Research](https://mlsanity.com/paper/2501.04306/) (2025)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [A Bayesian machine scientist to aid in the solution of challenging scientific problems](https://mlsanity.com/paper/2004.12157/) (2020)
- [SciDFM: A Large Language Model with Mixture-of-Experts for Science](https://mlsanity.com/paper/2409.18412/) (2024)
- [Approximation Methods for Partially Observed Markov Decision Processes (POMDPs)](https://mlsanity.com/paper/2108.13965/) (2021)

## Built on by (most cited first, 13 of 13)

- [From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery](https://mlsanity.com/paper/2505.13259/) (2025)
- [AstroVisBench: A Code Benchmark for Scientific Computing and Visualization in Astronomy](https://mlsanity.com/paper/2505.20538/) (2025)
- [ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers?](https://mlsanity.com/paper/2510.24591/) (2025)
- [DiscoverPhysics: Benchmarking LLMs for Out-of-the-Box Scientific Thinking](https://mlsanity.com/paper/2605.26087/) (2026)
- [Collider-Bench: Benchmarking AI Agents with Particle Physics Analysis Reproduction](https://mlsanity.com/paper/2605.13950/) (2026)
- [HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds](https://mlsanity.com/paper/2510.15614/) (2025)
- [Stargazer: A Scalable Model-Fitting Benchmark Environment for AI Agents under Astrophysical Constraints](https://mlsanity.com/paper/2604.15664/) (2026)
- [Can LLMs Discover Scientific Laws in Real and Parallel Worlds?](https://mlsanity.com/paper/2609.01552/) (2026)
- [Grounding LLMs in Scientific Discovery via Embodied Actions](https://mlsanity.com/paper/2602.20639/) (2026)
- [WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents](https://mlsanity.com/paper/2609.27490/) (2026)
- [The Marathon of Scientific Reasoning: Robustness of Scientific Agents to Perturbations in Multi-Turn Interactions](https://mlsanity.com/paper/2609.34537/) (2026)
- [EurekaBench: Measuring Agentic Ability to Discover New Scientific Insights](https://mlsanity.com/paper/2610.00492/) (2026)
- [LLM-IDEA: Identifiability-Driven Experimental Agent for Autonomous Discovery of Mechanistic World Models](https://mlsanity.com/paper/2610.11253/) (2026)

## Abstract (arXiv)

Modern science emerged from reasoning over repeatedly-observed planetary motions. We present Gravity-Bench-v1, an environment-based benchmark that challenges AI agents on tasks that parallel this historical development. Gravity-Bench-v1 evaluates agents on the discovery of physics concealed within a dynamic environment, using rigorous gravitational dynamics simulations. Gravity-Bench includes out-of-distribution cases, i.e. with physics that deviates from the real world, to evaluate true scientific generalization capabilities. Agents must plan to collect data within an experimental budget and must perform a dynamic form of data analysis and reasoning to solve tasks efficiently. Our benchmark admits an open-ended space of solutions. Reference solutions for each task are provided to calibrate AI performance against human expertise. Technically at an upper-undergraduate level, our benchmark proves challenging to baseline AI agents. Gravity-Bench-v1 and planned extensions should help map out AI progress towards scientific discovery capabilities.