mlsanityData to 7 Oct 2026

Dataset Distillation for Offline Reinforcement Learning

arXiv 2407.20299 ↗First posted 29 Jul 2024cs.LG cs.AIcited by 5

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline reinforcement learning often requires a quality dataset that we can train a policy on. However, in many situations, it is not possible to get such a dataset, nor is it easy to train a policy to perform well in the actual environment given the offline data. We propose using data distillation to train and distill a better dataset which can then be used for training a better policy model. We show that our method is able to synthesize a dataset where a model trained on it achieves similar performance to a model trained on the full dataset or a model trained using percentile behavioral cloning. Our project site is available at https://datasetdistillation4rl.github.io . We also provide our implementation at https://github.com/ggflow123/DDRL .

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 31 references that are in the map. First posted on arXiv 29 Jul 2024.

Builds on

23 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems2 Mar
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May

Other references · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  5. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  7. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  8. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
  9. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  10. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  11. 2022Dataset Distillation by Matching Training Trajectoriescited by 437
  12. 2015Policy Distillationcited by 414
9 more
  1. 2021Does Knowledge Distillation Really Work?cited by 187
  2. 2019Distilling Task-Specific Knowledge from BERT into Simple Neural Networkscited by 152
  3. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  4. 2023A Comprehensive Survey of Dataset Distillationcited by 114
  5. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  6. 2019Distilling Policy Distillationcited by 81
  7. 2020Reinforced Multi-Teacher Selection for Knowledge Distillationcited by 62
  8. 2019ChainerRL: A Deep Reinforcement Learning Librarycited by 51
  9. 2018Private Model Compression via Knowledge Distillationcited by 37

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

  1. 2024Provable Domain Adaptation for Offline Reinforcement Learning with Limited Samplescited by 5
  2. 2025Analytic Energy-Guided Policy Optimization for Offline Reinforcement Learningcited by 2
  3. 2025Distilling Reinforcement Learning into Single-Batch Datasetscited by 2
  4. 2025Decision Flow Policy Optimizationcited by 0
  5. 2025Algorithmic Guarantees for Distilling Supervised and Offline RL Datasetscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Offline Behavior Distillation
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning
  3. 2025Expert or not? assessing data quality in offline reinforcement learning
  4. 2022Data Valuation for Offline Reinforcement Learning
  5. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
  6. 2023Beyond Uniform Sampling: Offline Reinforcement Learning with Imbalanced Datasets
  7. 2020Offline Reinforcement Learning Hands-On
  8. 2021A Dataset Perspective on Offline Reinforcement Learning