Influence-Augmented Local Simulators: A Scalable Solution for Fast Deep RL in Large Networked Systems
Not on the map. Why?
Abstract (arXiv)
Learning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 23 of its 56 references that are in the map. First posted on arXiv 3 Feb 2022.
Builds on
23 of 56 references.
References · 23
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
- 2020Shortcut Learning in Deep Neural Networkscited by 1,928
- 2019Invariant Risk Minimizationcited by 1,832
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
- 2015Prioritized Experience Replaycited by 1,070
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
- 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
- 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
- 2019Challenges of Real-World Reinforcement Learningcited by 282
- 2019When to use parametric models in reinforcement learning?cited by 118
11 more
- 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61
- 2021Out of Distribution Generalization in Machine Learningcited by 59
- 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 52
- 2013Context-Specific Independence in Bayesian Networkscited by 43
- 2019Reinforcement Learning for Market Making in a Multi-agent Dealer Marketcited by 35
- 2019Learning Compact Models for Planning with Exogenous Processescited by 11
- 2019A Sufficient Statistic for Influence in Structured Multiagent Environmentscited by 8
- 2013An Algorithm for Finding Minimum d-Separating Sets in Belief Networkscited by 5
- 2020Loss Bounds for Approximate Influence-Based Abstractioncited by 4
- 2020Influence-Augmented Online Planning for Complex Environmentscited by 3
- 2021Offline Contextual Bandits for Wireless Network Optimizationcited by 1
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1.
2 papers.