mlsanityData to 14 Oct 2026

Dual-Camera All-in-Focus Neural Radiance Fields

arXiv 2504.16636 ↗First posted 23 Apr 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present the first framework capable of synthesizing the all-in-focus neural radiance field (NeRF) from inputs without manual refocusing. Without refocusing, the camera will automatically focus on the fixed object for all views, and current NeRF methods typically using one camera fail due to the consistent defocus blur and a lack of sharp reference. To restore the all-in-focus NeRF, we introduce the dual-camera from smartphones, where the ultra-wide camera has a wider depth-of-field (DoF) and the main camera possesses a higher resolution. The dual camera pair saves the high-fidelity details from the main camera and uses the ultra-wide camera's deep DoF as reference for all-in-focus restoration. To this end, we first implement spatial warping and color matching to align the dual camera, followed by a defocus-aware fusion module with learnable defocus parameters to predict a defocus map and fuse the aligned camera pair. We also build a multi-view dataset that includes image pairs of the main and ultra-wide cameras in a smartphone. Extensive experiments on this dataset verify that our solution, termed DC-NeRF, can produce high-quality all-in-focus novel views and compares favorably against strong baselines quantitatively and qualitatively. We further show DoF applications of DC-NeRF with adjustable blur intensity and focal plane, including refocusing and split diopter.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 68 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

43 of 68 references.

References · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  5. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  7. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  8. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  9. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  10. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  11. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 405
  12. 2021NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Imagescited by 336
31 more
  1. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  2. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  3. 2017Working hard to know your neighbor's margins: Local descriptor learning losscited by 212
  4. 2021Self-Calibrating Neural Radiance Fieldscited by 204
  5. 2021Deblur-NeRF: Neural Radiance Fields from Blurry Imagescited by 179
  6. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  7. 2021NeRF--: Neural Radiance Fields Without Known Camera Parameterscited by 155
  8. 2021GNeRF: GAN-based Neural Radiance Field without Posed Cameracited by 153
  9. 2022Learning Enriched Features for Fast Image Restoration and Enhancementcited by 150
  10. 2020Defocus Deblurring Using Dual-Pixel Datacited by 139
  11. 2021Iterative Filter Adaptive Network for Single Image Defocus Deblurringcited by 86
  12. 2018Synthetic Depth-of-Field with a Single-Camera Mobile Phonecited by 71
  13. 2022DP-NeRF: Deblurred Neural Radiance Field with Physical Scene Priorscited by 70
  14. 2016Unsupervised Learning of 3D Structure from Imagescited by 67
  15. 2020Rendering Natural Camera Bokeh Effect with Deep Learningcited by 57
  16. 2021Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutionscited by 54
  17. 2022BokehMe: When Neural Rendering Meets Classical Renderingcited by 48
  18. 2020Learning to Reduce Defocus Blur by Realistically Modeling Dual-Pixel Datacited by 42
  19. 2021DWDN: Deep Wiener Deconvolution Network for Non-Blind Image Deblurringcited by 36
  20. 2016Full Flow: Optical Flow Estimation By Global Optimization over Regular Gridscited by 34
  21. 2020Dual Pixel Exploration: Simultaneous Depth Estimation and Image Restorationcited by 34
  22. 2019NM-Net: Mining Reliable Neighbors for Robust Feature Correspondencescited by 29
  23. 2022DoF-NeRF: Depth-of-Field Meets Neural Radiance Fieldscited by 26
  24. 2017A Unified Approach of Multi-scale Deep and Hand-crafted Features for Defocus Estimationcited by 25
  25. 2017Aperture Supervision for Monocular Depth Estimationcited by 21
  26. 2020BGGAN: Bokeh-Glass Generative Adversarial Network for Rendering Realistic Bokehcited by 19
  27. 2023DC2: Dual-Camera Defocus Control by Learning to Refocuscited by 12
  28. 2020Depth-aware Blending of Smoothed Images for Bokeh Effect Generationcited by 10
  29. 2022NeRFocus: Neural Radiance Field for 3D Synthetic Defocuscited by 8
  30. 2023Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priorscited by 8
  31. 2023Point-and-Shoot All-in-Focus Photo Synthesis from Smartphone Camera Paircited by 4

Built on by

Not cited yet.