mlsanityData to 7 Oct 2026

Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning

arXiv 2302.06548 ↗First posted 13 Feb 2023cs.LG cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Tomorrow's robots will need to distinguish useful information from noise when performing different tasks. A household robot for instance may continuously receive a plethora of information about the home, but needs to focus on just a small subset to successfully execute its current chore. Filtering distracting inputs that contain irrelevant data has received little attention in the reinforcement learning literature. To start resolving this, we formulate a problem setting in reinforcement learning called the textitextremely noisy environment (ENE), where up to 99% of the input features are pure noise. Agents need to detect which features provide task-relevant information about the state of the environment. Consequently, we propose a new method termed textitAutomatic Noise Filtering (ANF), which uses the principles of dynamic sparse training in synergy with various deep reinforcement learning algorithms. The sparse input layer learns to focus its connectivity on task-relevant features, such that ANF-SAC and ANF-TD3 outperform standard SAC and TD3 by a large margin, while using up to 95% fewer weights. Furthermore, we devise a transfer learning setting for ENEs, by permuting all features of the environment after 1M timesteps to simulate the fact that other information sources can become relevant as the world evolves. Again, ANF surpasses the baselines in final performance and sample complexity. Our code is available at https://github.com/bramgrooten/automatic-noise-filtering

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

33 of 48 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  5. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  6. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  7. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  8. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 325
  9. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  10. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
  11. 2020The Hardware Lotterycited by 160
  12. 2017Deep Rewiring: Training very sparse deep networkscited by 136
21 more
  1. 2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 107
  2. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  3. 2021Continual Backprop: Stochastic Gradient Descent with Persistent Randomnesscited by 79
  4. 2020Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layerscited by 66
  5. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  6. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  7. 2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
  8. 2019Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardwarecited by 48
  9. 2022Sparsity Winning Twice: Better Robust Generalization from More Efficient Trainingcited by 41
  10. 2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
  11. 2020Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoderscited by 27
  12. 2021On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learningcited by 23
  13. 2021Single-Shot Pruning for Offline Reinforcement Learningcited by 21
  14. 2021Truly Sparse Neural Networks at Scalecited by 15
  15. 2022Where to Pay Attention in Sparse Training for Feature Selection?cited by 15
  16. 2020From Eye-blinks to State Construction: Diagnostic Benchmarks for Online Representation Learningcited by 13
  17. 2021Sparse Training Theory for Scalable and Efficient Agentscited by 12
  18. 2021Exploring the Training Robustness of Distributional Reinforcement Learning against Noisy State Observationscited by 11
  19. 2021Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spacescited by 9
  20. 2019On improving deep learning generalization with adaptive sparse connectivitycited by 7
  21. 2021Privacy Threats Analysis to Secure Federated Learningcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 6; 2025: 5.

13 papers.

Sort:

  1. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 38
  2. 2023Fantastic Weights and How to Find Them: Where to Prune in Dynamic Sparse Trainingcited by 16
  3. 2024Policy-shaped prediction: avoiding distractions in model-based reinforcement learningcited by 6
  4. 2025Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learningcited by 6
  5. 2025Reinitializing weights vs units for maintaining plasticity in neural networkscited by 6
  6. 2024Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidencecited by 5
  7. 2023Towards model-free RL algorithms that scale well with unstructured datacited by 3
  8. 2024TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learningcited by 3
  9. 2025On the Interplay Between Sparsity and Training in Deep Reinforcement Learningcited by 3
  10. 2024Hard-Thresholding Meets Evolution Strategies in Reinforcement Learningcited by 0
  11. 2024Boosting Robustness in Preference-Based Reinforcement Learning with Dynamic Sparsitycited by 0
  12. 2025NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensemblingcited by 0
1 more
  1. 2025Online Training and Pruning of Deep Reinforcement Learning Networkscited by 0