Super-Resolution Appearance Transfer for 4D Human Performances
Not on the map. Why?
Abstract (arXiv)
A common problem in the 4D reconstruction of people from multi-view video is the quality of the captured dynamic texture appearance which depends on both the camera resolution and capture volume. Typically the requirement to frame cameras to capture the volume of a dynamic performance (>50m3) results in the person occupying only a small proportion < 10% of the field of view. Even with ultra high-definition 4k video acquisition this results in sampling the person at less-than standard definition 0.5k video resolution resulting in low-quality rendering. In this paper we propose a solution to this problem through super-resolution appearance transfer from a static high-resolution appearance capture rig using digital stills cameras (> 8k) to capture the person in a small volume (<8m3). A pipeline is proposed for super-resolution appearance transfer from high-resolution static capture to dynamic video performance capture to produce super-resolution dynamic textures. This addresses two key problems: colour mapping between different camera systems; and dynamic texture map super-resolution using a learnt model. Comparative evaluation demonstrates a significant qualitative and quantitative improvement in rendering the 4D performance capture with super-resolution dynamic texture appearance. The proposed approach reproduces the high-resolution detail of the static capture whilst maintaining the appearance dynamics of the captured video.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 25 of its 66 references that are in the map. First posted on arXiv 31 Aug 2021.
Builds on
25 of 66 references.
References · 25
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
- 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,543
- 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
- 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
- 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
- 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
- 2019Multi-Garment Net: Learning to Dress 3D People from Imagescited by 244
- 2018Deep Back-Projection Networks For Super-Resolutioncited by 224
- 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
- 2019Feedback Network for Image Super-Resolutioncited by 155
13 more
- 2019Image Super-Resolution by Neural Texture Transfercited by 119
- 2018CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warpingcited by 90
- 2018DeepWrinkles: Accurate and Realistic Clothing Modelingcited by 90
- 2018Deep Exemplar-based Colorizationcited by 82
- 2020Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Imagescited by 65
- 2013Regularized Discrete Optimal Transportcited by 54
- 2017TextureGAN: Controlling Deep Image Synthesis with Texture Patchescited by 47
- 2018Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural networkcited by 44
- 2018Analyzing Perception-Distortion Tradeoff using Enhanced Perceptual Super-resolution Networkcited by 16
- 20193D Appearance Super-Resolution with Deep Learningcited by 11
- 2019Instance Segmentation based Semantic Matting for Compositing Applicationscited by 7
- 2020Learned Multi-View Texture Super-Resolutioncited by 7
- 2019Efficient Deep Neural Network for Photo-realistic Image Super-Resolutioncited by 5
Built on by
Citing papers per year: 2022: 1.
1 paper.