# CNN-based Patch Matching for Optical Flow with Thresholded Hinge Embedding Loss

arXiv: https://arxiv.org/abs/1607.08064 · HTML: https://mlsanity.com/paper/1607.08064/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jul 2016.

## Built on by (most cited first, 7 of 7)

- [Dense Continuous-Time Optical Flow from Events and Frames](https://mlsanity.com/paper/2203.13674/) (2022)
- [Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation](https://mlsanity.com/paper/2407.10802/) (2024)
- [SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Loss](https://mlsanity.com/paper/2109.15044/) (2021)
- [HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences](https://mlsanity.com/paper/2103.15573/) (2021)
- [FUNCTO: Function-Centric One-Shot Imitation Learning for Tool Manipulation](https://mlsanity.com/paper/2502.11744/) (2025)
- [Anchor-aware Deep Metric Learning for Audio-visual Retrieval](https://mlsanity.com/paper/2404.13789/) (2024)
- [Complementing Brightness Constancy with Deep Networks for Optical Flow Prediction](https://mlsanity.com/paper/2207.03790/) (2022)

## Abstract (arXiv)

Learning based approaches have not yet achieved their full potential in optical flow estimation, where their performance still trails heuristic approaches. In this paper, we present a CNN based patch matching approach for optical flow estimation. An important contribution of our approach is a novel thresholded loss for Siamese networks. We demonstrate that our loss performs clearly better than existing losses. It also allows to speed up training by a factor of 2 in our tests. Furthermore, we present a novel way for calculating CNN based features for different image scales, which performs better than existing methods. We also discuss new ways of evaluating the robustness of trained features for the application of patch matching for optical flow. An interesting discovery in our paper is that low-pass filtering of feature maps can increase the robustness of features created by CNNs. We proved the competitive performance of our approach by submitting it to the KITTI 2012, KITTI 2015 and MPI-Sintel evaluation portals where we obtained state-of-the-art results on all three datasets.