mlsanityData to 7 Oct 2026

Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging

arXiv 2204.09871 ↗First posted 21 Apr 2022cs.CV eess.IVcited by 6

Not on the map. Why?

Abstract (arXiv)

Cameras were originally designed using physics-based heuristics to capture aesthetic images. In recent years, there has been a transformation in camera design from being purely physics-driven to increasingly data-driven and task-specific. In this paper, we present a framework to understand the building blocks of this nascent field of end-to-end design of camera hardware and algorithms. As part of this framework, we show how methods that exploit both physics and data have become prevalent in imaging and computer vision, underscoring a key trend that will continue to dominate the future of task-specific camera design. Finally, we share current barriers to progress in end-to-end design, and hypothesize how these barriers can be overcome.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 47 of its 149 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

47 of 149 references.

References · 47

  1. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  2. 2021Convex optimizationcited by 2,479
  3. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  4. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  5. 2018Learning to See in the Darkcited by 459
  6. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  7. 2016Deep Convolutional Neural Network for Inverse Problems in Imagingcited by 431
  8. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  9. 2021Advances in Neural Renderingcited by 365
  10. 2019High Speed and High Dynamic Range Video with an Event Cameracited by 344
  11. 2020State of the Art on Neural Renderingcited by 276
  12. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
35 more
  1. 2018Deep Appearance Models for Face Renderingcited by 164
  2. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  3. 2021Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentationcited by 88
  4. 2018IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesiscited by 85
  5. 2019Deep Optics for Monocular Depth Estimation and 3D Object Detectioncited by 65
  6. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
  7. 2019Deep Optics for Single-shot High-dynamic-range Imagingcited by 62
  8. 2020Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profilescited by 59
  9. 2017DiffuserCam: Lensless Single-exposure 3D Imagingcited by 51
  10. 2021FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modellingcited by 47
  11. 2017Dirty Pixels: Towards End-to-End Image Processing and Perceptioncited by 42
  12. 2018Reblur2Deblur: Deblurring Videos via Self-Supervised Learningcited by 38
  13. 2022NeROIC: Neural Rendering of Objects from Online Image Collectionscited by 36
  14. 2018Geo-Supervised Visual Depth Predictioncited by 33
  15. 2019Learned reconstructions for practical mask-based lensless imagingcited by 31
  16. 2016Learning Sensor Multiplexing Design through Back-propagationcited by 28
  17. 2018Learning Depth from Single Images with Deep Neural Network Embedding Focal Lengthcited by 25
  18. 2020Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter arraycited by 24
  19. 2019Automatic ISP image quality tuning using non-linear optimizationcited by 20
  20. 2019Data-Driven Design for Fourier Ptychographic Microscopycited by 18
  21. 2020Miniscope3D: optimized single-shot miniature 3D fluorescence microscopycited by 17
  22. 2018DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imagingcited by 13
  23. 2017Convolutional neural networks that teach microscopes how to imagecited by 12
  24. 2022Towards Learning Neural Representations from Shadowscited by 12
  25. 2022Learning Spatially Varying Pixel Exposures for Motion Deblurringcited by 12
  26. 2019On Unlimited Sampling and Reconstructioncited by 11
  27. 2019Blending Diverse Physical Priors with Neural Networkscited by 11
  28. 2016Learning the image processing pipelinecited by 9
  29. 2019Feature Forwarding for Efficient Single Image Dehazingcited by 7
  30. 2020Fourier DiffuserScope: Single-shot 3D Fourier light field microscopy with a diffusercited by 7
  31. 2021DeepABM: Scalable, efficient and differentiable agent-based simulations via graph neural networkscited by 5
  32. 2021Deep Direct Volume Rendering: Learning Visual Feature Mappings From Exemplary Imagescited by 4
  33. 2016Shape from Mixed Polarizationcited by 3
  34. 2018Inverse Transport Networkscited by 2
  35. 2022Physically Disentangled Representationscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 0; 2025: 1; 2026: 1.

6 papers.

  1. 2023Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognitioncited by 20
  2. 2023Spatially Varying Exposure with 2-by-2 Multiplexing: Optimality and Universalitycited by 6
  3. 2023DISeR: Designing Imaging Systems with Reinforcement Learningcited by 6
  4. 2023Sensitivity analysis of AI-based algorithms for autonomous driving on optical wavefront aberrations induced by the windshieldcited by 2
  5. 2025γ-Quant: Towards Learnable Quantization for Low-bit Pattern Recognitioncited by 2
  6. 2026RAWDet-7: A Multi-Scenario Benchmark for Object Detection and Description on Quantized RAW Imagescited by 1