mlsanityData to 14 Oct 2026

PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning

arXiv 2306.10711 ↗First posted 19 Jun 2023cs.LGcited by 42

Not on the map. Why?

Cited by 24 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In Reinforcement Learning (RL), enhancing sample efficiency is crucial, particularly in scenarios when data acquisition is costly and risky. In principle, off-policy RL algorithms can improve sample efficiency by allowing multiple updates per environment interaction. However, these multiple updates often lead the model to overfit to earlier interactions, which is referred to as the loss of plasticity. Our study investigates the underlying causes of this phenomenon by dividing plasticity into two aspects. Input plasticity, which denotes the model's adaptability to changing input data, and label plasticity, which denotes the model's adaptability to evolving input-output relationships. Synthetic experiments on the CIFAR-10 dataset reveal that finding smoother minima of loss landscape enhances input plasticity, whereas refined gradient propagation improves label plasticity. Leveraging these findings, we introduce the PLASTIC algorithm, which harmoniously combines techniques to address both concerns. With minimal architectural modifications, PLASTIC achieves competitive performance on benchmarks including Atari-100k and Deepmind Control Suite. This result emphasizes the importance of preserving the model's plasticity to elevate the sample efficiency in RL. The code is available at https://github.com/dojeon-ai/plastic.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 64 of its 78 references that are in the map. First posted on arXiv 19 Jun 2023.

Builds on

64 of 78 references.

References · 64

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2016Layer Normalizationcited by 5,669
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  7. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  8. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  9. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  10. 2023Mastering Diverse Domains through World Modelscited by 1,253
  11. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  12. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
38 more
  1. 2015Prioritized Experience Replaycited by 1,099
  2. 2018DeepMind Control Suitecited by 956
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  4. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  6. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  9. 2019Model-Based Reinforcement Learning for Ataricited by 502
  10. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  11. 2020Reinforcement Learning with Augmented Datacited by 468
  12. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  13. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  14. 2022Transformers are Sample-Efficient World Modelscited by 328
  15. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  16. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  17. 2020Decoupling Representation Learning from Reinforcement Learningcited by 269
  18. 2017Noisy Networks for Explorationcited by 266
  19. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 260
  20. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  21. 2021Mastering Atari Games with Limited Datacited by 237
  22. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  23. 2023Understanding plasticity in neural networkscited by 174
  24. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 154
  25. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  26. 2022Understanding and Preventing Capacity Loss in Reinforcement Learningcited by 125
  27. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  28. 2019When to use parametric models in reinforcement learning?cited by 118
  29. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
  30. 2021Sharpness-Aware Minimization Improves Language Model Generalizationcited by 108
  31. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 108
  32. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  33. 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
  34. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  35. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 80
  36. 2020Deep Reinforcement and InfoMax Learningcited by 79
  37. 2022When Do Flat Minima Optimizers Work?cited by 78
  38. 2020Transient Non-Stationarity and Generalisation in Deep Reinforcement Learningcited by 75

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 17; 2025: 17; 2026: 6.

42 papers.

Sort:

By area:

  1. 2024DistiLLM: Towards Streamlined Distillation for Large Language Modelscited by 163
  2. 2024Disentangling the Causes of Plasticity Loss in Neural Networkscited by 86
  3. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learningcited by 76
  4. 2024Normalization and effective learning rates in reinforcement learningcited by 58
  5. 2024Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learningcited by 50
  6. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
  7. 2025Hyperspherical Normalization for Scalable Deep Reinforcement Learningcited by 38
  8. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 28
  9. 2024A Study of Plasticity Loss in On-Policy Deep Reinforcement Learningcited by 26
  10. 2025Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalizationcited by 19
  11. 2024Neuroplastic Expansion in Deep Reinforcement Learningcited by 18
  12. 2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RLcited by 16
30 more
  1. 2024Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Rewardcited by 16
  2. 2025Value-Based Deep RL Scales Predictablycited by 15
  3. 2024Can Learned Optimization Make Reinforcement Learning Less Difficult?cited by 14
  4. 2025Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agentscited by 12
  5. 2025Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learningcited by 11
  6. 2025What Can Grokking Teach Us About Learning Under Nonstationarity?cited by 8
  7. 2026Gradient Regularization Mitigates Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewardscited by 8
  8. 2024DASH: Warm-Starting Neural Network Training in Stationary Settings without Loss of Plasticitycited by 7
  9. 2025Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Losscited by 6
  10. 2025Reinitializing weights vs units for maintaining plasticity in neural networkscited by 6
  11. 2025Activation Function Design Sustains Plasticity in Continual Learningcited by 6
  12. 2026FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Controlcited by 5
  13. 2024Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learningcited by 4
  14. 2024Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learningcited by 4
  15. 2025A Champion-level Vision-based Reinforcement Learning Agent for Competitive Racing in Gran Turismo 7cited by 4
  16. 2025Flat Minima and Generalization: Insights from Stochastic Convex Optimizationcited by 4
  17. 2025Understanding Flatness in Generative Models: Its Role and Benefitscited by 3
  18. 2024Diverse Feature Learning by Self-distillation and Resetcited by 2
  19. 2024Continual Diffuser (CoD): Mastering Continual Offline Reinforcement Learning with Experience Rehearsalcited by 2
  20. 2025Plasticine: Accelerating Research in Plasticity-Motivated Deep Reinforcement Learningcited by 2
  21. 2026Do Neural Networks Lose Plasticity in a Gradually Changing World?cited by 2
  22. 2023Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularizationcited by 1
  23. 2025Surrogate-Assisted Evolutionary Reinforcement Learning Based on Autoencoder and Hyperbolic Neural Networkcited by 1
  24. 2025Scaling DRL for Decision Making: A Survey on Data, Network, and Training Budget Strategiescited by 1
  25. 2025Task Specific Sharpness Aware O-RAN Resource Management using Multi Agent Reinforcement Learningcited by 1
  26. 2026Can Scale Save Us From Plasticity Loss in Large Language Models?cited by 1
  27. 2024A Case for Validation Buffer in Pessimistic Actor-Criticcited by 0
  28. 2025Sharpness-Aware Minimization Can Hallucinate Minimizerscited by 0
  29. 2026Forager: a lightweight testbed for continual learning with partial observability in RLcited by 0
  30. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Controlcited by 0