ICo3D: An Interactive Conversational 3D Virtual Human
Not on the map. Why?
Abstract (arXiv)
This work presents Interactive Conversational 3D Virtual Human (ICo3D), a method for generating an interactive, conversational, and photorealistic 3D human avatar. Based on multi-view captures of a subject, we create an animatable 3D face model and a dynamic 3D body model, both rendered by splatting Gaussian primitives. Once merged together, they represent a lifelike virtual human avatar suitable for real-time user interactions. We equip our avatar with an LLM for conversational ability. During conversation, the audio speech of the avatar is used as a driving signal to animate the face model, enabling precise synchronization. We describe improvements to our dynamic Gaussian models that enhance photorealism: SWinGS++ for body reconstruction and HeadGaS++ for face reconstruction, and provide as well a solution to merge the separate face and body models without artifacts. We also present a demo of the complete system, showcasing several use cases of real-time conversation with the 3D avatar. Our approach offers a fully integrated virtual avatar experience, supporting both oral and written form interactions in immersive environments. ICo3D is applicable to a wide range of fields, including gaming, virtual assistance, and personalized education, among others. Project page: https://ico3d.github.io/
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 67 of its 87 references that are in the map. First posted on arXiv 19 Jan 2026.
Builds on
67 of 87 references.
References · 67
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2023Qwen Technical Reportcited by 3,273
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2023A Survey of Large Language Modelscited by 2,537
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
38 more
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
- 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2023A Survey on Evaluation of Large Language Modelscited by 1,227
- 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
- 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
- 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 753
- 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
- 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
- 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
- 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
- 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
- 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 470
- 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
- 2023Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesiscited by 341
- 2023SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenescited by 339
- 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
- 2018Deep Video Portraitscited by 265
- 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
- 2016Lip Reading Sentences in the Wildcited by 226
- 2023Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particlecited by 210
- 20233DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splattingcited by 195
- 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 190
- 2023GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussianscited by 190
- 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 183
- 2018Deep Appearance Models for Face Renderingcited by 164
- 2023HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motioncited by 161
- 2022Instant Volumetric Head Avatarscited by 157
- 20243DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videoscited by 156
- 2018Lip Movements Generation at a Glancecited by 154
- 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
- 2022Reconstructing Personalized Semantic Facial NeRF Models From Monocular Videocited by 128
- 2023Relightable Gaussian Codec Avatarscited by 126
- 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 107
- 2023Human Gaussian Splatting: Real-time Rendering of Animatable Avatarscited by 105
- 2023ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Renderingcited by 104
- 2023DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Renderingcited by 103
50 most cited of 67 All 67 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.