# Learning to Perform Physics Experiments via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1611.01843 · HTML: https://mlsanity.com/paper/1611.01843/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2016.

## Built on by (most cited first, 30 of 47)

- [REBEL: Reinforcement Learning via Regressing Relative Rewards](https://mlsanity.com/paper/2404.16767/) (2024)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents](https://mlsanity.com/paper/2305.02412/) (2023)
- [Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitation](https://mlsanity.com/paper/2107.01825/) (2021)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model](https://mlsanity.com/paper/2103.08255/) (2021)
- [CoMPS: Continual Meta Policy Search](https://mlsanity.com/paper/2112.04467/) (2021)
- [Comparing Popular Simulation Environments in the Scope of Robotics and Reinforcement Learning](https://mlsanity.com/paper/2103.04616/) (2021)
- [Gradient Shaping for Multi-Constraint Safe Reinforcement Learning](https://mlsanity.com/paper/2312.15127/) (2023)
- [PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing](https://mlsanity.com/paper/2407.16318/) (2024)
- [Hyperparameter Tuning for Deep Reinforcement Learning Applications](https://mlsanity.com/paper/2201.11182/) (2022)
- [Network Defense is Not a Game](https://mlsanity.com/paper/2104.10262/) (2021)
- [Explainability in Deep Reinforcement Learning, a Review into Current Methods and Applications](https://mlsanity.com/paper/2207.01911/) (2022)
- [Model-Free Active Exploration in Reinforcement Learning](https://mlsanity.com/paper/2407.00801/) (2024)
- [Combining machine learning with physics: A framework for tracking and sorting multiple dark solitons](https://mlsanity.com/paper/2111.04881/) (2021)
- [Innovations in Integrating Machine Learning and Agent-Based Modeling of Biomedical Systems](https://mlsanity.com/paper/2206.01092/) (2022)
- [On the Challenges of using Reinforcement Learning in Precision Drug Dosing: Delay and Prolongedness of Action Effects](https://mlsanity.com/paper/2301.00512/) (2023)
- [Stealing That Free Lunch: Exposing the Limits of Dyna-Style Reinforcement Learning](https://mlsanity.com/paper/2412.14312/) (2024)
- [A model-based approach to meta-Reinforcement Learning: Transformers and tree search](https://mlsanity.com/paper/2208.11535/) (2022)
- [A Deep Reinforcement Learning Based Motion Cueing Algorithm for Vehicle Driving Simulation](https://mlsanity.com/paper/2304.07600/) (2023)
- [Artificial intelligence, rationalization, and the limits of control in the public sector: the case of tax policy optimization](https://mlsanity.com/paper/2407.05336/) (2024)
- [Shifting Capsule Networks from the Cloud to the Deep Edge](https://mlsanity.com/paper/2110.02911/) (2021)
- [Reinforcement Learning-based Defect Mitigation for Quality Assurance of Additive Manufacturing](https://mlsanity.com/paper/2210.17272/) (2022)
- [Actively Learning Costly Reward Functions for Reinforcement Learning](https://mlsanity.com/paper/2211.13260/) (2022)
- [Multi-Agent Reinforcement Learning with Action Masking for UAV-enabled Mobile Communications](https://mlsanity.com/paper/2303.16737/) (2023)
- [A Review of Symbolic, Subsymbolic and Hybrid Methods for Sequential Decision Making](https://mlsanity.com/paper/2304.10590/) (2023)
- [Progressive-Resolution Policy Distillation: Leveraging Coarse-Resolution Simulations for Time-Efficient Fine-Resolution Policy Learning](https://mlsanity.com/paper/2412.07477/) (2024)
- [Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models](https://mlsanity.com/paper/2502.12776/) (2025)
- [Scan, Materialize, Simulate: A Generalizable Framework for Physically Grounded Robot Planning](https://mlsanity.com/paper/2505.14938/) (2025)
- [Parameter-free Gradient Temporal Difference Learning](https://mlsanity.com/paper/2105.04129/) (2021)

## Abstract (arXiv)

When encountering novel objects, humans are able to infer a wide range of physical properties such as mass, friction and deformability by interacting with them in a goal driven way. This process of active interaction is in the same spirit as a scientist performing experiments to discover hidden facts. Recent advances in artificial intelligence have yielded machines that can achieve superhuman performance in Go, Atari, natural language processing, and complex control problems; however, it is not clear that these systems can rival the scientific intuition of even a young child. In this work we introduce a basic set of tasks that require agents to estimate properties such as mass and cohesion of objects in an interactive simulated environment where they can manipulate the objects and observe the consequences. We found that state of art deep reinforcement learning methods can learn to perform the experiments necessary to discover such hidden properties. By systematically manipulating the problem difficulty and the cost incurred by the agent for performing experiments, we found that agents learn different strategies that balance the cost of gathering information against the cost of making mistakes in different situations.