mlsanityData to 14 Oct 2026

Transcending Dimensions using Generative AI: Real-Time 3D Model Generation in Augmented Reality

arXiv 2504.21033 ↗First posted 27 Apr 2025cs.GR cs.AI cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Traditional 3D modeling requires technical expertise, specialized software, and time-intensive processes, making it inaccessible for many users. Our research aims to lower these barriers by combining generative AI and augmented reality (AR) into a cohesive system that allows users to easily generate, manipulate, and interact with 3D models in real time, directly within AR environments. Utilizing cutting-edge AI models like Shap-E, we address the complex challenges of transforming 2D images into 3D representations in AR environments. Key challenges such as object isolation, handling intricate backgrounds, and achieving seamless user interaction are tackled through advanced object detection methods, such as Mask R-CNN. Evaluation results from 35 participants reveal an overall System Usability Scale (SUS) score of 69.64, with participants who engaged with AR/VR technologies more frequently rating the system significantly higher, at 80.71. This research is particularly relevant for applications in gaming, education, and AR-based e-commerce, offering intuitive, model creation for users without specialized skills.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 27 Apr 2025.

Builds on

24 of 36 references.

References · 24

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  2. 2017Mask R-CNNcited by 7,277
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  4. 2018YOLOv3: An Incremental Improvementcited by 2,659
  5. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  8. 2019Objects as Pointscited by 1,175
  9. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  10. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  11. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  12. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
12 more
  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  2. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  3. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  4. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  5. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  6. 2022Neural Wavelet-domain Diffusion for 3D Shape Generationcited by 132
  7. 2023Locally Attentional SDF Diffusion for Controllable 3D Shape Generationcited by 130
  8. 2022Transframer: Arbitrary Frame Prediction with Generative Modelscited by 37
  9. 2021CreativeGAN: Editing Generative Adversarial Networks for Creative Design Synthesiscited by 24
  10. 2023ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistencycited by 16
  11. 2019LassoNet: Deep Lasso-Selection of 3D Point Cloudscited by 10
  12. 2024Multilingual Standalone Trustworthy Voice-Based Social Network for Disaster Situationscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025When Generative Artificial Intelligence meets Extended Reality: A Systematic Reviewcited by 1
  2. 2026GEAR: GEometry-motion Alternating Refinement for Articulated Object Modeling with Gaussian Splattingcited by 1
  3. 2025Towards user-centered interactive medical image segmentation in VR with an assistive AI agentcited by 0
  4. 2026FAMOS: Feed-Forward 3D Articulation Modeling from Sparse Observationscited by 0