Learning to Look: Seeking Information for Decision Making via Policy Factorization
Not on the map. Why?
Abstract (arXiv)
Many robot manipulation tasks require active or interactive exploration behavior in order to be performed successfully. Such tasks are ubiquitous in embodied domains, where agents must actively search for the information necessary for each stage of a task, e.g., moving the head of the robot to find information relevant to manipulation, or in multi-robot domains, where one scout robot may search for the information that another robot needs to make informed decisions. We identify these tasks with a new type of problem, factorized Contextual Markov Decision Processes, and propose DISaM, a dual-policy solution composed of an information-seeking policy that explores the environment to find the relevant contextual information and an information-receiving policy that exploits the context to achieve the manipulation goal. This factorization allows us to train both policies separately, using the information-receiving one to provide reward to train the information-seeking policy. At test time, the dual agent balances exploration and exploitation based on the uncertainty the manipulation policy has on what the next best action is. We demonstrate the capabilities of our dual policy solution in five manipulation tasks that require information-seeking behaviors, both in simulation and in the real-world, where DISaM significantly outperforms existing methods. More information at https://robin-lab.cs.utexas.edu/learning2look/.
Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Builds on 42 of its 81 references that are in the map. First posted on arXiv 24 Oct 2024.
Builds on
42 of 81 references.
References · 42
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
- 2017Proximal Policy Optimization Algorithmscited by 14,592
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,022
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,223
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,028
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,450
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,368
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,016
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 857
- 2015Contextual Markov Decision Processescited by 215
- 2016Learning to Navigate in Complex Environmentscited by 193
- 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
30 more
- 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 155
- 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
- 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 112
- 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 98
- 2016Revisiting Active Perceptioncited by 91
- 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
- 2022Partially Observable Markov Decision Processes in Robotics: A Surveycited by 81
- 2016Interactive Perception: Leveraging Action in Perception and Perception in Actioncited by 77
- 2021RL for Latent MDPs: Regret Guarantees and a Lower Boundcited by 69
- 2019Online Replanning in Belief Space for Partially Observable Task and Motion Problemscited by 60
- 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
- 2013Sample Complexity of Multi-task Reinforcement Learningcited by 58
- 2021Partially Observable Markov Decision Processes (POMDPs) and Roboticscited by 49
- 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
- 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
- 2021Act the Part: Learning Interaction Strategies for Articulated Object Part Discoverycited by 45
- 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 27
- 2018Reinforcement Learning of Active Vision for Manipulating Objects under Occlusionscited by 25
- 2021Block Contextual MDPs for Continual Learningcited by 25
- 2022SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Renderingcited by 22
- 2013PEGASUS: A Policy Search Method for Large MDPs and POMDPscited by 21
- 2022Closed-Loop Next-Best-View Planning for Target-Driven Graspingcited by 19
- 2022Hypernetworks for Zero-shot Transfer in Reinforcement Learningcited by 18
- 2018Multi-View Picking: Next-best-view Reaching for Improved Grasping in Cluttercited by 15
- 2019Inverse Reinforcement Learning in Contextual MDPscited by 12
- 2023Asking Before Acting: Gather Information in Embodied Decision Making with Language Modelscited by 12
- 2020Active Perception and Representation for Robotic Manipulationcited by 10
- 2023Active-Perceptive Motion Generation for Mobile Manipulationcited by 10
- 2020Multi-Resolution POMDP Planning for Multi-Object Search in 3Dcited by 8
- 2023Language-Conditioned Observation Models for Visual Object Searchcited by 1
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.