# Content-aware Warping for View Synthesis

arXiv: https://arxiv.org/abs/2201.09023 · HTML: https://mlsanity.com/paper/2201.09023/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 64 references that are in the map.
- First posted on arXiv 22 Jan 2022.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Free View Synthesis](https://mlsanity.com/paper/2008.05511/) (2020)
- [Pushing the Boundaries of View Extrapolation with Multiplane Images](https://mlsanity.com/paper/1905.00413/) (2019)
- [Stable View Synthesis](https://mlsanity.com/paper/2011.07233/) (2020)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Extreme View Synthesis](https://mlsanity.com/paper/1812.04777/) (2018)
- [Learning to Synthesize a 4D RGBD Light Field from a Single Image](https://mlsanity.com/paper/1708.03292/) (2017)
- [High-dimensional Dense Residual Convolutional Neural Network for Light Field Reconstruction](https://mlsanity.com/paper/1910.01426/) (2019)
- [Light Field Reconstruction Using Shearlet Transform](https://mlsanity.com/paper/1509.08969/) (2015)
- [Towards a quality metric for dense light fields](https://mlsanity.com/paper/1704.07576/) (2017)
- [Self-Supervised Visibility Learning for Novel View Synthesis](https://mlsanity.com/paper/2103.15407/) (2021)
- [Revisiting Light Field Rendering with Deep Anti-Aliasing Neural Network](https://mlsanity.com/paper/2104.06797/) (2021)
- [Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide Baselines](https://mlsanity.com/paper/2108.07408/) (2021)

## Built on by (most cited first, 3 of 3)

- [Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression](https://mlsanity.com/paper/2307.06143/) (2023)
- [Enhancing Underwater Imaging with 4-D Light Fields: Dataset and Method](https://mlsanity.com/paper/2408.17339/) (2024)
- [Exploring Non-Local Spatial-Angular Correlations with a Hybrid Mamba-Transformer Framework for Light Field Super-Resolution](https://mlsanity.com/paper/2509.04824/) (2025)

## Abstract (arXiv)

Existing image-based rendering methods usually adopt depth-based image warping operation to synthesize novel views. In this paper, we reason the essential limitations of the traditional warping operation to be the limited neighborhood and only distance-based interpolation weights. To this end, we propose content-aware warping, which adaptively learns the interpolation weights for pixels of a relatively large neighborhood from their contextual information via a lightweight neural network. Based on this learnable warping module, we propose a new end-to-end learning-based framework for novel view synthesis from a set of input source views, in which two additional modules, namely confidence-based blending and feature-assistant spatial refinement, are naturally proposed to handle the occlusion issue and capture the spatial correlation among pixels of the synthesized view, respectively. Besides, we also propose a weight-smoothness loss term to regularize the network. Experimental results on light field datasets with wide baselines and multi-view datasets show that the proposed method significantly outperforms state-of-the-art methods both quantitatively and visually. The source code will be publicly available at https://github.com/MantangGuo/CW4VS.