mlsanityData to 7 Oct 2026

Training a Resilient Q-Network against Observational Interference

arXiv 2102.09677 ↗First posted 18 Feb 2021cs.LG cs.AI cs.NEcited by 9

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has demonstrated impressive performance in various gaming simulators and real-world applications. In practice, however, a DRL agent may receive faulty observation by abrupt interferences such as black-out, frozen-screen, and adversarial perturbation. How to design a resilient DRL algorithm against these rare but mission-critical and safety-crucial scenarios is an essential yet challenging task. In this paper, we consider a deep q-network (DQN) framework training with an auxiliary task of observational interferences such as artificial noises. Inspired by causal inference for observational interference, we propose a causal inference based DQN algorithm called causal inference Q-network (CIQ). We evaluate the performance of CIQ in several benchmark DQN environments with different types of interferences as auxiliary labels. Our experimental results show that the proposed CIQ method could achieve higher performance and more resilience against observational interferences.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 50 of its 84 references that are in the map. First posted on arXiv 18 Feb 2021.

Builds on

50 of 84 references.

References · 50

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  6. 2016OpenAI Gymcited by 1,684
  7. 2018World Modelscited by 1,444
  8. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  10. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  11. 2017Safe Reinforcement Learning via Shieldingcited by 427
  12. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 408
38 more
  1. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  2. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  3. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  4. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  5. 2017Adversarial Attacks on Neural Network Policiescited by 314
  6. 2019Learning Latent Plans from Playcited by 310
  7. 2018Towards a Definition of Disentangled Representationscited by 294
  8. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  9. 2017Noisy Networks for Explorationcited by 262
  10. 2018Unity: A General Platform for Intelligent Agentscited by 232
  11. 2016Learning to Navigate in Complex Environmentscited by 192
  12. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 181
  13. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  14. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  15. 2017Visualizing and Understanding Atari Agentscited by 141
  16. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  17. 2019Adversarial Training Can Hurt Generalizationcited by 131
  18. 2019Behaviour Suite for Reinforcement Learningcited by 129
  19. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
  20. 2018Deep Variational Reinforcement Learning for POMDPscited by 122
  21. 2020Invariant Causal Prediction for Block MDPscited by 104
  22. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
  23. 2020Causal Autoregressive Flowscited by 97
  24. 2017Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigationcited by 90
  25. 2013Deep Learning of Representations: Looking Forwardcited by 84
  26. 2019Off-Policy Evaluation in Partially Observable Environmentscited by 65
  27. 2020A Causal View on Robustness of Neural Networkscited by 53
  28. 2018Temporal Difference Variational Auto-Encodercited by 48
  29. 2020Off-policy Evaluation in Infinite-Horizon Reinforcement Learning with Latent Confounderscited by 40
  30. 2018Deconfounding Reinforcement Learning in Observational Settingscited by 36
  31. 2013On the Testability of Causal Models with Latent and Instrumental Variablescited by 26
  32. 2020Multi-task Language Modeling for Improving Speech Recognition of Rare Wordscited by 21
  33. 2020Enhanced Adversarial Strategically-Timed Attacks against Deep Reinforcement Learningcited by 13
  34. 2020Counterfactually Guided Off-policy Transfer in Clinical Settingscited by 13
  35. 2018Bayesian Policy Optimization for Model Uncertaintycited by 12
  36. 2019When Causal Intervention Meets Adversarial Examples and Image Masking for Deep Neural Networkscited by 7
  37. 2019Transfer in Deep Reinforcement Learning using Knowledge Graphscited by 5
  38. 2020Improving Fair Predictions Using Variational Inference In Causal Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 2; 2025: 1; 2026: 1.

9 papers.

  1. 2023Explainable Reinforcement Learning via a Causal World Modelcited by 22
  2. 2023Causal Reinforcement Learning: A Surveycited by 22
  3. 2024From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichmentcited by 8
  4. 2023Causal Video Summarizer for Video Explorationcited by 4
  5. 2023Q-Cogni: An Integrated Causal Reinforcement Learning Frameworkcited by 2
  6. 2025Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defensescited by 2
  7. 2024Personalized Video Summarization using Text-Based Queries and Conditional Modelingcited by 1
  8. 2022Inference and Denoise: Causal Inference-based Neural Speech Enhancementcited by 0
  9. 2026Convex Low-resource Accent-Robust Language Detection in Speech Recognitioncited by 0