mlsanityData to 7 Oct 2026

Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football

arXiv 2303.15471 ↗First posted 25 Mar 2023cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Artificial Intelligence has been used to help human complete difficult tasks in complicated environments by providing optimized strategies for decision-making or replacing the manual labour. In environments including multiple agents, such as football, the most common methods to train agents are Imitation Learning and Multi-Agent Reinforcement Learning (MARL). However, the agents trained by Imitation Learning cannot outperform the expert demonstrator, which makes humans hardly get new insights from the learnt policy. Besides, MARL is prone to the credit assignment problem. In environments with sparse reward signal, this method can be inefficient. The objective of our research is to create a novel reward shaping method by embedding contextual information in reward function to solve the aforementioned challenges. We demonstrate this in the Google Research Football (GRF) environment. We quantify the contextual information extracted from game state observation and use this quantification together with original sparse reward to create the shaped reward. The experiment results in the GRF environment prove that our reward shaping method is a useful addition to state-of-the-art MARL algorithms for training agents in environments with sparse reward signal.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 28 references that are in the map. First posted on arXiv 25 Mar 2023.

Builds on

9 of 28 references.

References · 9

  1. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  2. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  3. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  4. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  5. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  6. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 112
  7. 2017Coordinated Multi-Agent Imitation Learningcited by 70
  8. 2021Self-Supervised Online Reward Shaping in Sparse-Reward Environmentscited by 34
  9. 2019Reward Shaping via Meta-Learningcited by 30

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021From Motor Control to Team Play in Simulated Humanoid Football
  2. 2025Generative World Models of Tasks: LLM-Driven Hierarchical Scaffolding for Embodied Agents
  3. 2020Reward Shaping for Human Learning via Inverse Reinforcement Learning
  4. 2021MimicBot: Combining Imitation and Reinforcement Learning to win in Bot Bowl
  5. 2024MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning
  6. 2026Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback
  7. 2021Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Data
  8. 2019Reinforced Imitation in Heterogeneous Action Space