# SpikeMM: Flexi-Magnification of High-Speed Micro-Motions

arXiv: https://arxiv.org/abs/2406.00383 · HTML: https://mlsanity.com/paper/2406.00383/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 57 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 20 of 20)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [High-Quality Self-Supervised Deep Image Denoising](https://mlsanity.com/paper/1901.10277/) (2019)
- [Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPs](https://mlsanity.com/paper/2111.15135/) (2021)
- [Deep Learning on Implicit Neural Representations of Shapes](https://mlsanity.com/paper/2302.05438/) (2023)
- [Learning-based Video Motion Magnification](https://mlsanity.com/paper/1804.02684/) (2018)
- [A Retina-inspired Sampling Method for Visual Texture Reconstruction](https://mlsanity.com/paper/1907.08769/) (2019)
- [Optical Flow Estimation for Spiking Camera](https://mlsanity.com/paper/2110.03916/) (2021)
- [1000x Faster Camera and Machine Vision with Ordinary Devices](https://mlsanity.com/paper/2201.09302/) (2022)
- [Video Acceleration Magnification](https://mlsanity.com/paper/1704.04186/) (2017)
- [Regularize implicit neural representation by itself](https://mlsanity.com/paper/2303.15484/) (2023)
- [Self-Supervised Motion Magnification by Backpropagating Through Optical Flow](https://mlsanity.com/paper/2311.17056/) (2023)
- [Lagrangian Motion Magnification with Double Sparse Optical Flow Decomposition](https://mlsanity.com/paper/2204.07636/) (2022)
- [SpikeCV: Open a Continuous Computer Vision Era](https://mlsanity.com/paper/2303.11684/) (2023)
- [Towards the Next Generation of Retinal Neuroprosthesis: Visual Computation with Spikes](https://mlsanity.com/paper/2001.04064/) (2020)
- [Unveiling the Potential of Spike Streams for Foreground Occlusion Removal from Densely Continuous Views](https://mlsanity.com/paper/2307.00821/) (2023)
- [Revealing Fine Structures of the Retinal Receptive Field by Deep Learning Networks](https://mlsanity.com/paper/1811.02290/) (2018)

## Built on by (most cited first, 1 of 1)

- [USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting](https://mlsanity.com/paper/2411.10504/) (2024)

## Abstract (arXiv)

The amplification of high-speed micro-motions holds significant promise, with applications spanning fault detection in fast-paced industrial environments to refining precision in medical procedures. However, conventional motion magnification algorithms often encounter challenges in high-speed scenarios due to low sampling rates or motion blur. In recent years, spike cameras have emerged as a superior alternative for visual tasks in such environments, owing to their unique capability to capture temporal and spatial frequency domains with exceptional fidelity. Unlike conventional cameras, which operate at fixed, low frequencies, spike cameras emulate the functionality of the retina, asynchronously capturing photon changes at each pixel position using spike streams. This innovative approach comprehensively records temporal and spatial visual information, rendering it particularly suitable for magnifying high-speed micro-motions.This paper introduces SpikeMM, a pioneering spike-based algorithm tailored specifically for high-speed motion magnification. SpikeMM integrates multi-level information extraction, spatial upsampling, and motion magnification modules, offering a self-supervised approach adaptable to a wide range of scenarios. Notably, SpikeMM facilitates seamless integration with high-performance super-resolution and motion magnification algorithms. We substantiate the efficacy of SpikeMM through rigorous validation using scenes captured by spike cameras, showcasing its capacity to magnify motions in real-world high-frequency settings.