mlsanityData to 7 Oct 2026

SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies

arXiv 2106.09678 ↗First posted 17 Jun 2021cs.LG cs.AI cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Generalization has been a long-standing challenge for reinforcement learning (RL). Visual RL, in particular, can be easily distracted by irrelevant factors in high-dimensional observation space. In this work, we consider robust policy learning which targets zero-shot generalization to unseen visual environments with large distributional shift. We propose SECANT, a novel self-expert cloning technique that leverages image augmentation in two stages to decouple robust representation learning from policy optimization. Specifically, an expert policy is first trained by RL from scratch with weak augmentations. A student network then learns to mimic the expert policy by supervised learning with strong augmentations, making its representation more robust against visual variations compared to the expert. Extensive experiments demonstrate that SECANT significantly advances the state of the art in zero-shot generalization across 4 challenging domains. Our average reward improvements over prior SOTAs are: DeepMind Control (+26.5%), robotic manipulation (+337.8%), vision-based autonomous driving (+47.7%), and indoor object navigation (+15.8%). Code release and video are available at https://linxifan.github.io/secant-site/.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 65 of its 77 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

65 of 77 references.

References · 65

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  10. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  11. 2016Generative Adversarial Imitation Learningcited by 1,576
  12. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
38 more
  1. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2018DeepMind Control Suitecited by 935
  5. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  7. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  8. 2018Learning Dexterous In-Hand Manipulationcited by 643
  9. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  10. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  11. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  12. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  13. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  14. 2020Reinforcement Learning with Augmented Datacited by 452
  15. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  16. 2015Policy Distillationcited by 414
  17. 2016Learning to reinforcement learncited by 369
  18. 2019Learning by Cheatingcited by 361
  19. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  20. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  21. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  22. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  23. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  24. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  25. 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 231
  26. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
  27. 2020Prioritized Level Replaycited by 190
  28. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  29. 2017Visualizing and Understanding Atari Agentscited by 141
  30. 2020RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environmentscited by 135
  31. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  32. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 132
  33. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  34. 2018Generalization and Regularization in DQNcited by 117
  35. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 102
  36. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  37. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  38. 2018Playing hard exploration games by watching YouTubecited by 98

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 12; 2023: 15; 2024: 4; 2025: 8; 2026: 2.

42 papers.

Sort:

By area:

  1. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  2. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  3. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  4. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 171
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  6. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  7. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 58
  8. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  9. 2023What Makes Pre-Trained Visual Representations Successful for Robust Manipulation?cited by 38
  10. 2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37
  11. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 32
  12. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 27
30 more
  1. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 24
  2. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  3. 2022Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learningcited by 21
  4. 2022Learning Task-relevant Representations for Generalization via Characteristic Functions of Reward Sequence Distributionscited by 15
  5. 2022Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racingcited by 13
  6. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13
  7. 2023Normalization Enhances Generalization in Visual Reinforcement Learningcited by 13
  8. 2024A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learningcited by 12
  9. 2023Sample Efficient Reinforcement Learning in Mixed Systems through Augmented Samples and Its Applications to Queueing Networkscited by 9
  10. 2023MaDi: Learning to Mask Distractions for Generalization in Visual Deep Reinforcement Learningcited by 9
  11. 2023Policy Gradient Methods in the Presence of Symmetries and State Abstractionscited by 8
  12. 2023Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learningcited by 8
  13. 2023Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertaintycited by 8
  14. 2023Visual-Policy Learning through Multi-Camera View to Single-Camera View Knowledge Distillation for Robot Manipulation Taskscited by 7
  15. 2023Symmetric Replay Training: Enhancing Sample Efficiency in Deep Reinforcement Learning for Combinatorial Optimizationcited by 6
  16. 2024Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidencecited by 5
  17. 2023Generalization in Visual Reinforcement Learning with the Reward Sequence Distributioncited by 4
  18. 2025SLIM: Sim-to-Real Legged Instructive Manipulation via Long-Horizon Visuomotor Learningcited by 4
  19. 2025See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrationscited by 4
  20. 2022MoDA: Map style transfer for self-supervised Domain Adaptation of embodied agentscited by 3
  21. 2025Dream to Generalize: Zero-Shot Model-Based Reinforcement Learning for Unseen Visual Distractionscited by 3
  22. 2025Merging and Disentangling Views in Visual Reinforcement Learning for Robotic Manipulationcited by 2
  23. 2025Self-Predictive Dynamics for Generalization of Vision-based Reinforcement Learningcited by 2
  24. 2024Efficient Training of Generalizable Visuomotor Policies via Control-Aware Augmentationcited by 1
  25. 2025Salience-Invariant Consistent Policy Learning for Generalization in Visual Reinforcement Learningcited by 1
  26. 2025How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learningcited by 1
  27. 2022The Sandbox Environment for Generalizable Agent Research (SEGAR)cited by 0
  28. 2025Semore: VLM-guided Enhanced Semantic Motion Representations for Visual Reinforcement Learningcited by 0
  29. 2026Inductive Generalization for Robotic Manipulationcited by 0
  30. 2026Task-Relevant Representation Decoupling for Visual Reinforcement Learning Generalizationcited by 0