mlsanityData to 7 Oct 2026

Learning Task Informed Abstractions

arXiv 2106.15612 ↗First posted 29 Jun 2021cs.LG cs.AI cs.ROcited by 58

Not on the map. Why?

Abstract (arXiv)

Current model-based reinforcement learning methods struggle when operating from complex visual scenes due to their inability to prioritize task-relevant features. To mitigate this problem, we propose learning Task Informed Abstractions (TIA) that explicitly separates reward-correlated visual features from distractors. For learning TIA, we introduce the formalism of Task Informed MDP (TiMDP) that is realized by training two models that learn visual features via cooperative reconstruction, but one model is adversarially dissociated from the reward signal. Empirical evaluation shows that TIA leads to significant performance gains over state-of-the-art methods on many visual control tasks where natural and unconstrained visual distractions pose a formidable challenge.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 55 of its 67 references that are in the map. First posted on arXiv 29 Jun 2021.

Builds on

55 of 67 references.

References · 55

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  5. 2017The Kinetics Human Action Video Datasetcited by 1,894
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  8. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  9. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  10. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  11. 2020Mastering Atari with Discrete World Modelscited by 990
  12. 2018DeepMind Control Suitecited by 935
38 more
  1. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  4. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  5. 2019Model-Based Reinforcement Learning for Ataricited by 495
  6. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  7. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  8. 2020Reinforcement Learning with Augmented Datacited by 452
  9. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  10. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
  11. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  12. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  13. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  14. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  15. 2016Variational Intrinsic Controlcited by 262
  16. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  17. 2019Learning De-biased Representations with Biased Representationscited by 199
  18. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 183
  19. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  20. 2016Value Iteration Networkscited by 158
  21. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  22. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  23. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 147
  24. 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 137
  25. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  26. 2019Unlearn Dataset Bias in Natural Language Inference by Fitting the Residualcited by 119
  27. 2017Value Prediction Networkcited by 117
  28. 2020Invariant Causal Prediction for Block MDPscited by 104
  29. 2015Learning to See by Movingcited by 101
  30. 2021The Distracting Control Suite -- A Challenging Benchmark for Reinforcement Learning from Pixelscited by 97
  31. 2020Counterfactual Data Augmentation using Locally Factored Dynamicscited by 93
  32. 2016The Predictron: End-To-End Learning and Planningcited by 92
  33. 2013Deep Learning of Representations: Looking Forwardcited by 84
  34. 2020Deep Reinforcement and InfoMax Learningcited by 79
  35. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  36. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  37. 2020Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictorscited by 60
  38. 2020Predictive Information Accelerates Learning in RLcited by 56

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 9; 2023: 17; 2024: 13; 2025: 6; 2026: 11.

58 papers.

Sort:

By area:

  1. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  2. 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 66
  3. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 64
  4. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  5. 2022Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Modelscited by 42
  6. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 41
  7. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  8. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
  9. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  10. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  11. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  12. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
38 more
  1. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  2. 2021Learning Representations for Pixel-based Control: What Matters and Why?cited by 24
  3. 2023Learning World Models with Identifiable Factorizationcited by 23
  4. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  5. 2024Learning Latent Dynamic Robust Representations for World Modelscited by 20
  6. 2023RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictabilitycited by 17
  7. 2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
  8. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13
  9. 2023Generalizable Visual Reinforcement Learning with Segment Anything Modelcited by 12
  10. 2024Building Minimal and Reusable Causal State Abstractions for Reinforcement Learningcited by 11
  11. 2022Task-Induced Representation Learningcited by 10
  12. 2023Ignorance is Bliss: Robust Control via Information Gatingcited by 10
  13. 2023SeMAIL: Eliminating Distractors in Visual Imitation via Separated Modelscited by 10
  14. 2024AD3: Implicit Action is the Key for World Models to Distinguish the Diverse Visual Distractorscited by 10
  15. 2023Combining Reconstruction and Contrastive Methods for Multimodal Representations in RLcited by 9
  16. 2024Policy-shaped prediction: avoiding distractions in model-based reinforcement learningcited by 6
  17. 2025Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Controlcited by 6
  18. 2023Task Aware Dreamer for Task Generalization in Reinforcement Learningcited by 5
  19. 2024Focus On What Matters: Separated Models For Visual-Based RL Generalizationcited by 5
  20. 2023Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learningcited by 4
  21. 2024Make the Pertinent Salient: Task-Relevant Reconstruction for Visual Control with Distractionscited by 4
  22. 2026Temporally Centered SIGReg Improves LeWorldModel Representations for Robot Policy Learningcited by 4
  23. 2022Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part Icited by 3
  24. 2023Sequential Action-Induced Invariant Representation for Reinforcement Learningcited by 3
  25. 2024BECAUSE: Bilinear Causal Representation for Generalizable Offline Model-based Reinforcement Learningcited by 3
  26. 2024Physically Interpretable World Models via Weakly Supervised Representation Learningcited by 3
  27. 2022Disentangled (Un)Controllable Featurescited by 2
  28. 2026Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modelingcited by 2
  29. 2026DWM: Separating World Effects from Actions in Latent World Modelscited by 2
  30. 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistencycited by 2
  31. 2024Efficient Training of Generalizable Visuomotor Policies via Control-Aware Augmentationcited by 1
  32. 2024SeMOPO: Learning High-quality Model and Policy from Low-quality Offline Visual Datasetscited by 1
  33. 2025Learning Fused State Representations for Control from Multi-View Observationscited by 1
  34. 2023Towards Control-Centric Representations in Reinforcement Learning from Imagescited by 0
  35. 2024Rethinking State Disentanglement in Causal Reinforcement Learningcited by 0
  36. 2024Problem Space Transformations for Out-of-Distribution Generalisation in Behavioural Cloningcited by 0
  37. 2025Self-Consistent Model-based Adaptation for Visual Reinforcement Learningcited by 0
  38. 2025MInCo: Mitigating Information Conflicts in Distracted Visual Model-based Reinforcement Learningcited by 0

50 most cited of 58 All 58 (JSON)