# Federated Multi-View Synthesizing for Metaverse

arXiv: https://arxiv.org/abs/2401.00859 · HTML: https://mlsanity.com/paper/2401.00859/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 50 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 25 of 25)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [SplitFed: When Federated Learning Meets Split Learning](https://mlsanity.com/paper/2004.12088/) (2020)
- [EpiGRAF: Rethinking training of 3D GANs](https://mlsanity.com/paper/2206.10535/) (2022)
- [Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation](https://mlsanity.com/paper/2202.13162/) (2022)
- [Compressing Volumetric Radiance Fields to 1 MB](https://mlsanity.com/paper/2211.16386/) (2022)
- [Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs](https://mlsanity.com/paper/2011.00844/) (2020)
- [Sliced Wasserstein Distance for Learning Gaussian Mixture Models](https://mlsanity.com/paper/1711.05376/) (2017)
- [Generative Multiplane Images: Making a 2D GAN 3D-Aware](https://mlsanity.com/paper/2207.10642/) (2022)
- [SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields](https://mlsanity.com/paper/2212.02501/) (2022)
- [4K-NeRF: High Fidelity Neural Radiance Fields at Ultra High Resolutions](https://mlsanity.com/paper/2212.04701/) (2022)
- [When Internet of Things meets Metaverse: Convergence of Physical and Cyber Worlds](https://mlsanity.com/paper/2208.13501/) (2022)
- [Generalized One-shot Domain Adaptation of Generative Adversarial Networks](https://mlsanity.com/paper/2209.03665/) (2022)
- [A Robust Deep Learning Enabled Semantic Communication System for Text](https://mlsanity.com/paper/2206.02596/) (2022)
- [Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis](https://mlsanity.com/paper/2207.10257/) (2022)
- [Learning-based Prediction, Rendering and Transmission for Interactive Virtual Reality in RIS-Assisted Terahertz Networks](https://mlsanity.com/paper/2107.12943/) (2021)
- [Semantic Communications with Variable-Length Coding for Extended Reality](https://mlsanity.com/paper/2302.08645/) (2023)
- [User Dynamics-Aware Edge Caching and Computing for Mobile Virtual Reality](https://mlsanity.com/paper/2311.10645/) (2023)
- [Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning](https://mlsanity.com/paper/2107.01001/) (2021)
- [CoVRage: Millimeter-Wave Beamforming for Mobile Interactive Virtual Reality](https://mlsanity.com/paper/2212.05865/) (2022)
- [Content Popularity Prediction Based on Quantized Federated Bayesian Learning in Fog Radio Access Networks](https://mlsanity.com/paper/2206.12258/) (2022)

## Built on by (most cited first, 4 of 4)

- [Towards Agentic Intelligence for Materials Science](https://mlsanity.com/paper/2602.00169/) (2026)
- [Personalized Federated Learning for Cellular VR: Online Learning and Dynamic Caching](https://mlsanity.com/paper/2501.11745/) (2025)
- [A Lightweight Human Pose Estimation Approach for Edge Computing-Enabled Metaverse with Compressive Sensing](https://mlsanity.com/paper/2409.00087/) (2024)
- [Decentralized Fairness Aware Multi Task Federated Learning for VR Network](https://mlsanity.com/paper/2512.02513/) (2025)

## Abstract (arXiv)

The metaverse is expected to provide immersive entertainment, education, and business applications. However, virtual reality (VR) transmission over wireless networks is data- and computation-intensive, making it critical to introduce novel solutions that meet stringent quality-of-service requirements. With recent advances in edge intelligence and deep learning, we have developed a novel multi-view synthesizing framework that can efficiently provide computation, storage, and communication resources for wireless content delivery in the metaverse. We propose a three-dimensional (3D)-aware generative model that uses collections of single-view images. These single-view images are transmitted to a group of users with overlapping fields of view, which avoids massive content transmission compared to transmitting tiles or whole 3D models. We then present a federated learning approach to guarantee an efficient learning process. The training performance can be improved by characterizing the vertical and horizontal data samples with a large latent feature space, while low-latency communication can be achieved with a reduced number of transmitted parameters during federated learning. We also propose a federated transfer learning framework to enable fast domain adaptation to different target domains. Simulation results have demonstrated the effectiveness of our proposed federated multi-view synthesizing framework for VR content delivery.