# DeepMPCVS: Deep Model Predictive Control for Visual Servoing

arXiv: https://arxiv.org/abs/2105.00788 · HTML: https://mlsanity.com/paper/2105.00788/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2021.

## Built on by (most cited first, 3 of 3)

- [ObjectReact: Learning Object-Relative Control for Visual Navigation](https://mlsanity.com/paper/2509.09594/) (2025)
- [WM-VS: Progress-Aligned World Models for Closed-Loop Visual Servoing](https://mlsanity.com/paper/2609.20892/) (2026)
- [VGM-VS: Rethinking Visual Geometry Model for High-Precision Visual Servoing](https://mlsanity.com/paper/2609.28312/) (2026)

## Abstract (arXiv)

The simplicity of the visual servoing approach makes it an attractive option for tasks dealing with vision-based control of robots in many real-world applications. However, attaining precise alignment for unseen environments pose a challenge to existing visual servoing approaches. While classical approaches assume a perfect world, the recent data-driven approaches face issues when generalizing to novel environments. In this paper, we aim to combine the best of both worlds. We present a deep model predictive visual servoing framework that can achieve precise alignment with optimal trajectories and can generalize to novel environments. Our framework consists of a deep network for optical flow predictions, which are used along with a predictive model to forecast future optical flow. For generating an optimal set of velocities we present a control network that can be trained on the fly without any supervision. Through extensive simulations on photo-realistic indoor settings of the popular Habitat framework, we show significant performance gain due to the proposed formulation vis-a-vis recent state-of-the-art methods. Specifically, we show a faster convergence and an improved performance in trajectory length over recent approaches.