# Motion-from-Blur: 3D Shape and Motion Estimation of Motion-blurred Objects in Videos

arXiv: https://arxiv.org/abs/2111.14465 · HTML: https://mlsanity.com/paper/2111.14465/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better](https://mlsanity.com/paper/1908.03826/) (2019)
- [Softmax Splatting for Video Frame Interpolation](https://mlsanity.com/paper/2003.05534/) (2020)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Deblurring by Realistic Blurring](https://mlsanity.com/paper/2004.01860/) (2020)
- [Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring](https://mlsanity.com/paper/2004.05343/) (2020)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research](https://mlsanity.com/paper/1911.05063/) (2019)
- [Cascaded Deep Video Deblurring Using Temporal Sharpness Prior](https://mlsanity.com/paper/2004.02501/) (2020)
- [Deep Animation Video Interpolation in the Wild](https://mlsanity.com/paper/2104.02495/) (2021)
- [Learning to Extract a Video Sequence from a Single Motion-Blurred Image](https://mlsanity.com/paper/1804.04065/) (2018)
- [Blurry Video Frame Interpolation](https://mlsanity.com/paper/2002.12259/) (2020)
- [CDFI: Compression-Driven Network Design for Frame Interpolation](https://mlsanity.com/paper/2103.10559/) (2021)
- [ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring](https://mlsanity.com/paper/2103.04260/) (2021)
- [Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers](https://mlsanity.com/paper/1804.10975/) (2018)
- [Deblurring using Analysis-Synthesis Networks Pair](https://mlsanity.com/paper/2004.02956/) (2020)
- [DeFMO: Deblurring and Shape Recovery of Fast Moving Objects](https://mlsanity.com/paper/2012.00595/) (2020)
- [Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments](https://mlsanity.com/paper/2103.16095/) (2021)
- [The World of Fast Moving Objects](https://mlsanity.com/paper/1611.07889/) (2016)
- [Intra-frame Object Tracking by Deblatting](https://mlsanity.com/paper/1905.03633/) (2019)
- [Sub-frame Appearance and 6D Pose Estimation of Fast Moving Objects](https://mlsanity.com/paper/1911.10927/) (2019)
- [Shape from Blur: Recovering Textured 3D Shape and Motion of Fast Moving Objects](https://mlsanity.com/paper/2106.08762/) (2021)
- [FMODetect: Robust Detection of Fast Moving Objects](https://mlsanity.com/paper/2012.08216/) (2020)

## Built on by (most cited first, 11 of 11)

- [Blur Interpolation Transformer for Real-World Motion from Blur](https://mlsanity.com/paper/2211.11423/) (2022)
- [Restore Anything Pipeline: Segment Anything Meets Image Restoration](https://mlsanity.com/paper/2305.13093/) (2023)
- [Tracking by 3D Model Estimation of Unknown Objects in Videos](https://mlsanity.com/paper/2304.06419/) (2023)
- [GS-Blur: A 3D Scene-Based Dataset for Realistic Image Deblurring](https://mlsanity.com/paper/2410.23658/) (2024)
- [Human from Blur: Human Pose Tracking from Blurry Images](https://mlsanity.com/paper/2303.17209/) (2023)
- [Deblur Gaussian Splatting SLAM](https://mlsanity.com/paper/2503.12572/) (2025)
- [Where Is The Ball: 3D Ball Trajectory Estimation From 2D Monocular Tracking](https://mlsanity.com/paper/2506.05763/) (2025)
- [Video frame interpolation for high dynamic range sequences captured with dual-exposure sensors](https://mlsanity.com/paper/2206.09485/) (2022)
- [Retrieval Robust to Object Motion Blur](https://mlsanity.com/paper/2404.18025/) (2024)
- [UPGS: Unified Pose-aware Gaussian Splatting for Dynamic Scene Deblurring](https://mlsanity.com/paper/2509.00831/) (2025)
- [Recovering 3D Shapes from Ultra-Fast Motion-Blurred Images](https://mlsanity.com/paper/2602.07860/) (2026)

## Abstract (arXiv)

We propose a method for jointly estimating the 3D motion, 3D shape, and appearance of highly motion-blurred objects from a video. To this end, we model the blurred appearance of a fast moving object in a generative fashion by parametrizing its 3D position, rotation, velocity, acceleration, bounces, shape, and texture over the duration of a predefined time window spanning multiple frames. Using differentiable rendering, we are able to estimate all parameters by minimizing the pixel-wise reprojection error to the input video via backpropagating through a rendering pipeline that accounts for motion blur by averaging the graphics output over short time intervals. For that purpose, we also estimate the camera exposure gap time within the same optimization. To account for abrupt motion changes like bounces, we model the motion trajectory as a piece-wise polynomial, and we are able to estimate the specific time of the bounce at sub-frame accuracy. Experiments on established benchmark datasets demonstrate that our method outperforms previous methods for fast moving object deblurring and 3D reconstruction.