mlsanityData to 14 Oct 2026

Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributions

arXiv 2205.10218 ↗First posted 20 May 2022cs.LG cs.AI cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Generalization across different environments with the same tasks is critical for successful applications of visual reinforcement learning (RL) in real scenarios. However, visual distractions -- which are common in real scenes -- from high-dimensional observations can be hurtful to the learned representations in visual RL, thus degrading the performance of generalization. To tackle this problem, we propose a novel approach, namely Characteristic Reward Sequence Prediction (CRESP), to extract the task-relevant information by learning reward sequence distributions (RSDs), as the reward signals are task-relevant in RL and invariant to visual distractions. Specifically, to effectively capture the task-relevant information via RSDs, CRESP introduces an auxiliary task -- that is, predicting the characteristic functions of RSDs -- to learn task-relevant representations, because we can well approximate the high-dimensional distributions by leveraging the corresponding characteristic functions. Experiments demonstrate that CRESP significantly improves the performance of generalization on unseen environments, outperforming several state-of-the-arts on DeepMind Control tasks with different visual distractions.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

29 of 36 references.

References · 29

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  5. 2018DeepMind Control Suitecited by 956
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  8. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  9. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  10. 2020Reinforcement Learning with Augmented Datacited by 468
  11. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  12. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
17 more
  1. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  2. 2020Self-Supervised Policy Adaptation during Deploymentcited by 123
  3. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  4. 2018Generalization and Regularization in DQNcited by 118
  5. 2020Invariant Causal Prediction for Block MDPscited by 105
  6. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 104
  7. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  8. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 100
  9. 2019Observational Overfitting in Reinforcement Learningcited by 91
  10. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  11. 2017Towards Generalization and Simplicity in Continuous Controlcited by 68
  12. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 43
  13. 2020Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learningcited by 37
  14. 2019A Characteristic Function Approach to Deep Implicit Generative Modelingcited by 26
  15. 2020Rotation, Translation, and Cropping for Zero-Shot Generalizationcited by 24
  16. 2021Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Criticcited by 23
  17. 2021Invariant Policy Learning: A Causal Perspectivecited by 17

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 7; 2024: 4; 2025: 3.

15 papers.

Sort:

By area:

  1. 2023De Novo Molecular Generation via Connection-aware Motif Miningcited by 46
  2. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 25
  3. 2024KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulationcited by 17
  4. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  5. 2025Apollo-MILP: An Alternating Prediction-Correction Neural Solving Framework for Mixed-Integer Linear Programmingcited by 16
  6. 2024Scalable and Effective Arithmetic Tree Generation for Adder and Multiplier Designscited by 9
  7. 2024Learning to Stop Cut Generation for Efficient Mixed-Integer Linear Programmingcited by 8
  8. 2023Road Planning for Slums via Deep Reinforcement Learningcited by 7
  9. 2023Accelerate Presolve in Large-Scale Linear Programming via Reinforcement Learningcited by 7
  10. 2023State Sequences Prediction via Fourier Transform for Representation Learningcited by 7
  11. 2023Generalization in Visual Reinforcement Learning with the Reward Sequence Distributioncited by 4
  12. 2023Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learningcited by 4
3 more
  1. 2023Sequential Action-Induced Invariant Representation for Reinforcement Learningcited by 3
  2. 2025DRARL: Disengagement-Reason-Augmented Reinforcement Learning for Efficient Improvement of Autonomous Driving Policycited by 1
  3. 2024Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applicationscited by 0