mlsanityData to 14 Oct 2026

The Indoor-Training Effect: unexpected gains from distribution shifts in the transition function

arXiv 2401.15856 ↗First posted 29 Jan 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Is it better to perform tennis training in a pristine indoor environment or a noisy outdoor one? To model this problem, here we investigate whether shifts in the transition probabilities between the training and testing environments in reinforcement learning problems can lead to better performance under certain conditions. We generate new Markov Decision Processes (MDPs) starting from a given MDP, by adding quantifiable, parametric noise into the transition function. We refer to this process as Noise Injection and the resulting environments as δ-environments. This process allows us to create variations of the same environment with quantitative control over noise serving as a metric of distance between environments. Conventional wisdom suggests that training and testing on the same MDP should yield the best results. In stark contrast, we observe that agents can perform better when trained on the noise-free environment and tested on the noisy δ-environments, compared to training and testing on the same δ-environments. We confirm that this finding extends beyond noise variations: it is possible to showcase the same phenomenon in ATARI game variations including varying Ghost behaviour in PacMan, and Paddle behaviour in Pong. We demonstrate this intriguing behaviour across 60 different variations of ATARI games, including PacMan, Pong, and Breakout. We refer to this phenomenon as the Indoor-Training Effect. Code to reproduce our experiments and to implement Noise Injection can be found at https://bit.ly/3X6CTYk.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 38 references that are in the map. First posted on arXiv 29 Jan 2024.

Builds on

20 of 38 references.

References · 20

  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  2. 2019Solving Rubik's Cube with a Robot Handcited by 734
  3. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  4. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  5. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  6. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 336
  7. 2019Challenges of Real-World Reinforcement Learningcited by 291
  8. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  9. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  10. 2018Assessing Generalization in Deep Reinforcement Learningcited by 133
  11. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 100
  12. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 81
8 more
  1. 2019Simulation-based reinforcement learning for real-world autonomous drivingcited by 52
  2. 2017Deep Object-Centric Representations for Generalizable Robot Learningcited by 50
  3. 2019Structured agents for physical constructioncited by 50
  4. 2021The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priorscited by 41
  5. 2019Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flightcited by 28
  6. 2021When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenescited by 23
  7. 2020Distributionally Robust Markov Decision Processes and their Connection to Risk Measurescited by 14
  8. 2021In-distribution adversarial attacks on object recognition models using gradient-free searchcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025STORI: A Benchmark and Taxonomy for Stochastic Environments
  2. 2026On Training in Imagination
  3. 2025Reinforcement Learning via Implicit Imitation Guidance
  4. 2021A Closer Look at Advantage-Filtered Behavioral Cloning in High-Noise Datasets
  5. 2026Player-1: Learning to Play from Synthetic Experience
  6. 2026Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning
  7. 2025Offline Reinforcement Learning with Penalized Action Noise Injection
  8. 2026Offline-Online Reinforcement Learning for Linear Mixture MDPs