# Synthetically Generating Human-like Data for Sequential Decision Making Tasks via Reward-Shaped Imitation Learning

arXiv: https://arxiv.org/abs/2304.07280 · HTML: https://mlsanity.com/paper/2304.07280/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 14 Apr 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 15 of 15)

- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [A Survey of Data Augmentation Approaches for NLP](https://mlsanity.com/paper/2105.03075/) (2021)
- [Data Augmentation Generative Adversarial Networks](https://mlsanity.com/paper/1711.04340/) (2017)
- [GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification](https://mlsanity.com/paper/1803.01229/) (2018)
- [Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping](https://mlsanity.com/paper/2011.02669/) (2020)
- [BAGAN: Data Augmentation with Balancing GAN](https://mlsanity.com/paper/1803.09655/) (2018)
- [Differentially Private Synthetic Medical Data Generation using Convolutional GANs](https://mlsanity.com/paper/2012.11774/) (2020)
- [TrajGAIL: Generating Urban Vehicle Trajectories using Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2007.14189/) (2020)
- [imitation: Clean Imitation Learning Implementations](https://mlsanity.com/paper/2211.11972/) (2022)
- [Improving the Accuracy of Global Forecasting Models using Time Series Data Augmentation](https://mlsanity.com/paper/2008.02663/) (2020)
- [Generative Adversarial Networks for Image Augmentation in Agriculture: A Systematic Review](https://mlsanity.com/paper/2204.04707/) (2022)
- [Why GANs are overkill for NLP](https://mlsanity.com/paper/2205.09838/) (2022)

## Abstract (arXiv)

We consider the problem of synthetically generating data that can closely resemble human decisions made in the context of an interactive human-AI system like a computer game. We propose a novel algorithm that can generate synthetic, human-like, decision making data while starting from a very small set of decision making data collected from humans. Our proposed algorithm integrates the concept of reward shaping with an imitation learning algorithm to generate the synthetic data. We have validated our synthetic data generation technique by using the synthetically generated data as a surrogate for human interaction data to solve three sequential decision making tasks of increasing complexity within a small computer game-like setup. Different empirical and statistical analyses of our results show that the synthetically generated data can substitute the human data and perform the game-playing tasks almost indistinguishably, with very low divergence, from a human performing the same tasks.