mlsanityData to 14 Oct 2026

VirtualCube: An Immersive 3D Video Communication System

arXiv 2112.06730 ↗First posted 13 Dec 2021cs.CV cs.GR cs.MMcited by 11

Not on the map. Why?

Abstract (arXiv)

The VirtualCube system is a 3D video conference system that attempts to overcome some limitations of conventional technologies. The key ingredient is VirtualCube, an abstract representation of a real-world cubicle instrumented with RGBD cameras for capturing the 3D geometry and texture of a user. We design VirtualCube so that the task of data capturing is standardized and significantly simplified, and everything can be built using off-the-shelf hardware. We use VirtualCubes as the basic building blocks of a virtual conferencing environment, and we provide each VirtualCube user with a surrounding display showing life-size videos of remote participants. To achieve real-time rendering of remote participants, we develop the V-Cube View algorithm, which uses multi-view stereo for more accurate depth estimation and Lumi-Net rendering for better rendering quality. The VirtualCube system correctly preserves the mutual eye gaze between participants, allowing them to establish eye contact and be aware of who is visually paying attention to them. The system also allows a participant to have side discussions with remote participants as if they were in the same room. Finally, the system sheds lights on how to support the shared space of work items (e.g., documents and applications) and track the visual attention of participants to work items.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 31 of its 91 references that are in the map. First posted on arXiv 13 Dec 2021.

Builds on

31 of 91 references.

References · 31

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  4. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  6. 2016Least Squares Generative Adversarial Networkscited by 1,001
  7. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  8. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  9. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
  10. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
  11. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
  12. 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 320
19 more
  1. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  2. 2018Deep Video Portraitscited by 267
  3. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  4. 2019Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Losscited by 252
  5. 2018Talking Face Generation by Adversarially Disentangled Audio-Visual Representationcited by 212
  6. 2019Few-shot Video-to-Video Synthesiscited by 197
  7. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  8. 2016Learning-Based View Synthesis for Light Field Camerascited by 145
  9. 2020Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatarscited by 118
  10. 2020Neural Head Reenactment with Latent Pose Descriptorscited by 106
  11. 2020Background Matting: The World is Your Green Screencited by 105
  12. 2019Text-based Editing of Talking-head Videocited by 103
  13. 2018Extreme View Synthesiscited by 94
  14. 2019MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targetscited by 90
  15. 2021Driving-Signal Aware Full-Body Avatarscited by 81
  16. 2020Deep 3D Portrait from a Single Imagecited by 38
  17. 2019FLNet: Landmark Driven Fetching and Learning Network for Faithful Talking Facial Animation Synthesiscited by 28
  18. 2016Supervised Transformer Network for Efficient Face Detectioncited by 14
  19. 2021Egocentric Videoconferencingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 2; 2025: 0; 2026: 2.

11 papers.

  1. 2022Streaming Radiance Fields for 3D Video Synthesiscited by 119
  2. 2022SteerNeRF: Accelerating NeRF Rendering via Smooth Viewpoint Trajectorycited by 15
  3. 2023Learning Visibility Field for Detailed 3D Human Reconstruction and Relightingcited by 11
  4. 2024HiCoM: Hierarchical Coherent Motion for Streamable Dynamic Scene with 3D Gaussian Splattingcited by 6
  5. 2022CU-Net: Real-Time High-Fidelity Color Upsampling for Point Cloudscited by 2
  6. 2023RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touchcited by 2
  7. 2023RealLiFe: Real-Time Light Field Reconstruction via Hierarchical Sparse Gradient Descentcited by 1
  8. 2026Mon3tr: Monocular 3D Telepresence with Pre-built Gaussian Avatars as Amortizationcited by 1
  9. 2023Bringing Telepresence to Every Deskcited by 0
  10. 2024Thing2Reality: Transforming 2D Content into Conditioned Multiviews and 3D Gaussian Objects for XR Communicationcited by 0
  11. 2026Transformer-Based Inpainting for Real-Time 3D Streaming in Sparse Multi-Camera Setupscited by 0