mlsanityData to 7 Oct 2026

MaRINeR: Enhancing Novel Views by Matching Rendered Images with Nearby References

arXiv 2407.13745 ↗First posted 18 Jul 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Rendering realistic images from 3D reconstruction is an essential task of many Computer Vision and Robotics pipelines, notably for mixed-reality applications as well as training autonomous agents in simulated environments. However, the quality of novel views heavily depends of the source reconstruction which is often imperfect due to noisy or missing geometry and appearance. Inspired by the recent success of reference-based super-resolution networks, we propose MaRINeR, a refinement method that leverages information of a nearby mapping image to improve the rendering of a target viewpoint. We first establish matches between the raw rendered image of the scene geometry from the target viewpoint and the nearby reference based on deep features, followed by hierarchical detail transfer. We show improved renderings in quantitative metrics and qualitative examples from both explicit and implicit scene representations. We further employ our method on the downstream tasks of pseudo-ground-truth validation, synthetic data enhancement and detail recovery for renderings of reduced 3D reconstructions.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

44 of 71 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  9. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  10. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  12. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
32 more
  1. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  2. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  3. 2020Convolutional Occupancy Networkscited by 760
  4. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  5. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  6. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  7. 2017Universal Style Transfer via Feature Transformscited by 352
  8. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
  9. 2018The relativistic discriminator: a key element missing from standard GANcited by 234
  10. 2017Neural Style Transfer: A Reviewcited by 199
  11. 2019Photorealistic Style Transfer via Wavelet Transformscited by 166
  12. 2019Image Super-Resolution by Neural Texture Transfercited by 119
  13. 2021ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flowscited by 113
  14. 2022LaMAR: Benchmarking Localization and Mapping for Augmented Realitycited by 98
  15. 2018CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warpingcited by 90
  16. 2021MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolutioncited by 79
  17. 2021Robust Reference-based Super-Resolution via C2-Matchingcited by 68
  18. 2019Revealing Scenes by Inverting Structure from Motion Reconstructionscited by 66
  19. 2023Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labelscited by 64
  20. 2016Learning to Navigate the Energy Landscapecited by 62
  21. 2022Reference-based Image Super-Resolution with Deformable Attention Transformercited by 50
  22. 2019Multimodal Style Transfer via Graph Cutscited by 32
  23. 2020Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenescited by 31
  24. 2023NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXercited by 29
  25. 2024EgoGen: An Egocentric Synthetic Data Generatorcited by 27
  26. 2023Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlookcited by 24
  27. 2021ERQA: Edge-Restoration Quality Assessment for Video Super-Resolutioncited by 14
  28. 2021Attention-based Multi-Reference Learning for Image Super-Resolutioncited by 12
  29. 2022RRSR:Reciprocal Reference-based Image Super-Resolution with Progressive Feature Alignment and Selectioncited by 12
  30. 2023LMR: A Large-Scale Multi-Reference Dataset for Reference-based Super-Resolutioncited by 12
  31. 2023A Feature Reuse Framework with Texture-adaptive Aggregation for Reference-based Super-Resolutioncited by 2
  32. 2022Multi-Reference Image Super-Resolution: A Posterior Fusion Approachcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026One-Shot Refiner: Boosting Feed-forward Novel View Synthesis via One-Step Diffusioncited by 4
  2. 2025SRSplat: Feed-Forward Super-Resolution Gaussian Splatting from Sparse Multi-View Imagescited by 2
  3. 2026Improving Local Feature Matching by Entropy-inspired Scale Adaptability and Flow-endowed Local Consistencycited by 0
  4. 2026UniFixer: A Universal Reference-Guided Fixer for Diffusion-Based View Synthesiscited by 0