# ICPRL: Acquiring Physical Intuition from Interactive Control

arXiv: https://arxiv.org/abs/2603.13295 · HTML: https://mlsanity.com/paper/2603.13295/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Latent predictive world models, one of 11 directions in World models.
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 1 Mar 2026.

## Placed in

- World models / [Latent predictive world models](https://mlsanity.com/wm/latent-predictive-world-models/)

## Builds on (most cited first, 11 of 11)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents](https://mlsanity.com/paper/2310.09971/) (2023)
- [Training a Generally Curious Agent](https://mlsanity.com/paper/2502.17543/) (2025)
- [Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling](https://mlsanity.com/paper/2502.00622/) (2025)
- [Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skills](https://mlsanity.com/paper/2503.12533/) (2025)
- [DeepPHY: Benchmarking Agentic VLMs on Physical Reasoning](https://mlsanity.com/paper/2508.05405/) (2025)
- [Model-Based Decentralized Policy Optimization](https://mlsanity.com/paper/2302.08139/) (2023)
- [NS-Gym: Open-Source Simulation Environments and Benchmarks for Non-Stationary Markov Decision Processes](https://mlsanity.com/paper/2501.09646/) (2025)
- [SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios](https://mlsanity.com/paper/2511.17649/) (2025)

## Built on by (most cited first, 1 of 1)

- [In-Context Reinforcement Learning under Non-Stationarity: A Survey](https://mlsanity.com/paper/2607.11906/) (2026)

## Abstract (arXiv)

VLMs excel at static perception but falter in interactive reasoning in dynamic physical environments, which demands planning and adaptation to dynamic outcomes. Existing physical reasoning methods often depend on abstract symbolic inputs or lack the ability to learn and adapt from direct, pixel-based visual interaction in novel scenarios. We introduce ICPRL (In-Context Physical Reinforcement Learning), a framework inspired by In-Context Reinforcement Learning (ICRL) that empowers VLMs to acquire physical intuition and adapt their policies in-context. Our approach trains a vision-grounded policy model via multi-turn Group Relative Policy Optimization (GRPO) over diverse multi-episode interaction histories. This enables the agent to adapt strategies by conditioning on past trial-and-error sequences, without requiring any weight updates. This adaptive policy works in concert with a separately trained world model that provides explicit physical reasoning by predicting the results of potential actions. At inference, the policy proposes candidate actions, while the world model predicts outcomes to guide a root-node PUCT search to select the most promising action. Evaluated on the diverse physics-based puzzle-solving tasks in the DeepPHY benchmark, ICPRL demonstrates significant improvements across both its I. policy-only, and II. world-model-augmented stages. Notably, these gains are retained in unseen physical environments, demonstrating that our framework facilitates genuine in-context acquisition of the environment's physical dynamics from interactive experience.