# Lagrangian Motion Magnification with Double Sparse Optical Flow Decomposition

arXiv: https://arxiv.org/abs/2204.07636 · HTML: https://mlsanity.com/paper/2204.07636/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 12 of 12)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [Deep Multimodal Learning for Audio-Visual Speech Recognition](https://mlsanity.com/paper/1501.05396/) (2015)
- [Disentangling Architecture and Training for Optical Flow](https://mlsanity.com/paper/2203.10712/) (2022)
- [Video Acceleration Magnification](https://mlsanity.com/paper/1704.04186/) (2017)
- [Optical Flow Techniques for Facial Expression Analysis -- a Practical Evaluation Study](https://mlsanity.com/paper/1904.11592/) (2019)
- [First order algorithms in variational image processing](https://mlsanity.com/paper/1412.4237/) (2014)
- [Vibration-Based Damage Detection in Wind Turbine Blades using Phase-Based Motion Estimation and Motion Magnification](https://mlsanity.com/paper/1804.00558/) (2018)
- [Self-Supervised Approach for Facial Movement Based Optical Flow](https://mlsanity.com/paper/2105.01256/) (2021)

## Built on by (most cited first, 7 of 7)

- [Self-Supervised Motion Magnification by Backpropagating Through Optical Flow](https://mlsanity.com/paper/2311.17056/) (2023)
- [T-FAKE: Synthesizing Thermal Images for Facial Landmarking](https://mlsanity.com/paper/2408.15127/) (2024)
- [SpikeMM: Flexi-Magnification of High-Speed Micro-Motions](https://mlsanity.com/paper/2406.00383/) (2024)
- [Towards Reliable Sea Ice Drift Estimation in the Arctic Deep Learning Optical Flow on RADARSAT-2](https://mlsanity.com/paper/2510.26653/) (2025)
- [EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Control](https://mlsanity.com/paper/2507.15292/) (2025)
- [GeoDiffMM: Geometry-Guided Conditional Diffusion for Motion Magnification](https://mlsanity.com/paper/2512.08325/) (2025)
- [HOT-POT: Optimal Transport for Sparse Stereo Matching](https://mlsanity.com/paper/2601.12423/) (2026)

## Abstract (arXiv)

Microexpressions are fast and spatially small facial expressions that are difficult to detect. Therefore motion magnification techniques, which aim at amplifying and hence revealing subtle motion in videos, appear useful for handling such expressions. There are basically two main approaches, namely via Eulerian or Lagrangian techniques. While the first one magnifies motion implicitly by operating directly on image pixels, the Lagrangian approach uses optical flow (OF) techniques to extract and magnify pixel trajectories. In this paper, we propose a novel approach for local Lagrangian motion magnification of facial micro-motions. Our contribution is three-fold: first, we fine tune the recurrent all-pairs field transforms (RAFT) for OFs deep learning approach for faces by adding ground truth obtained from the variational dense inverse search (DIS) for OF algorithm applied to the CASME II video set of facial micro expressions. This enables us to produce OFs of facial videos in an efficient and sufficiently accurate way. Second, since facial micro-motions are both local in space and time, we propose to approximate the OF field by sparse components both in space and time leading to a double sparse decomposition. Third, we use this decomposition to magnify micro-motions in specific areas of the face, where we introduce a new forward warping strategy using a triangular splitting of the image grid and barycentric interpolation of the RGB vectors at the corners of the transformed triangles. We demonstrate the feasibility of our approach by various examples.