# Learning Sparse Control Tasks from Pixels by Latent Nearest-Neighbor-Guided Explorations

arXiv: https://arxiv.org/abs/2302.14242 · HTML: https://mlsanity.com/paper/2302.14242/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2023.

## Built on by (most cited first, 4 of 4)

- [DynaGuide: Steering Diffusion Polices with Active Dynamic Guidance](https://mlsanity.com/paper/2506.13922/) (2025)
- [Task-aware Distributed Source Coding under Dynamic Bandwidth](https://mlsanity.com/paper/2305.15523/) (2023)
- [Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrations](https://mlsanity.com/paper/2412.01114/) (2024)
- [Expert Behavior Prior Reinforcement Learning](https://mlsanity.com/paper/2607.21302/) (2026)

## Abstract (arXiv)

Recent progress in deep reinforcement learning (RL) and computer vision enables artificial agents to solve complex tasks, including locomotion, manipulation and video games from high-dimensional pixel observations. However, domain specific reward functions are often engineered to provide sufficient learning signals, requiring expert knowledge. While it is possible to train vision-based RL agents using only sparse rewards, additional challenges in exploration arise. We present a novel and efficient method to solve sparse-reward robot manipulation tasks from only image observations by utilizing a few demonstrations. First, we learn an embedded neural dynamics model from demonstration transitions and further fine-tune it with the replay buffer. Next, we reward the agents for staying close to the demonstrated trajectories using a distance metric defined in the embedding space. Finally, we use an off-policy, model-free vision RL algorithm to update the control policies. Our method achieves state-of-the-art sample efficiency in simulation and enables efficient training of a real Franka Emika Panda manipulator.