mlsanityData to 14 Oct 2026

Control4D: Efficient 4D Portrait Editing with Text

arXiv 2305.20082 ↗First posted 31 May 2023cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

We introduce Control4D, an innovative framework for editing dynamic 4D portraits using text instructions. Our method addresses the prevalent challenges in 4D editing, notably the inefficiencies of existing 4D representations and the inconsistent editing effect caused by diffusion-based editors. We first propose GaussianPlanes, a novel 4D representation that makes Gaussian Splatting more structured by applying plane-based decomposition in 3D space and time. This enhances both efficiency and robustness in 4D editing. Furthermore, we propose to leverage a 4D generator to learn a more continuous generation space from inconsistent edited images produced by the diffusion-based editor, which effectively improves the consistency and quality of 4D editing. Comprehensive evaluation demonstrates the superiority of Control4D, including significantly reduced training time, high-quality rendering, and spatial-temporal consistency in 4D portrait editing. The link to our project website is https://control4darxiv.github.io.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 66 of its 91 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

66 of 91 references.

References · 66

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2022Generative Adversarial Networkscited by 5,627
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  11. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  3. 2022Video Diffusion Modelscited by 2,305
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  8. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  12. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  13. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  14. 2021Variational Diffusion Modelscited by 1,153
  15. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  16. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  17. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  18. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  19. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  20. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  21. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  22. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  23. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  24. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  25. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  26. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  27. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  28. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  29. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  30. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  31. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  32. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  33. 2021Advances in Neural Renderingcited by 365
  34. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  35. 2023Pix2Video: Video Editing using Image Diffusioncited by 320
  36. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  37. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  38. 2021Function4D: Real-time Human Volumetric Capture from Very Sparse Consumer RGBD Sensorscited by 246

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 12; 2025: 11; 2026: 5.

32 papers.

Sort:

By area:

  1. 2024A Survey on 3D Gaussian Splattingcited by 224
  2. 2023GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussianscited by 191
  3. 2023DynMF: Neural Motion Factorization for Real-time Dynamic View Synthesis with 3D Gaussian Splattingcited by 124
  4. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 78
  5. 2024GaussianObject: High-Quality 3D Object Reconstruction from Four Views with Gaussian Splattingcited by 59
  6. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 41
  7. 2023HHAvatar: Gaussian Head Avatar with Dynamic Hairscited by 35
  8. 2024SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decompositioncited by 30
  9. 2024LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priorscited by 25
  10. 2024GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Viewscited by 22
  11. 2023LatentEditor: Text Driven Local Editing of 3D Scenescited by 17
  12. 2024HDRSplat: Gaussian Splatting for High Dynamic Range 3D Scene Reconstruction from Raw Imagescited by 15
20 more
  1. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 15
  2. 2024Portrait Video Editing Empowered by Multimodal Generative Priorscited by 12
  3. 2024CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusioncited by 11
  4. 2025UV-Attack: Physical-World Adversarial Attacks for Person Detection via Dynamic-NeRF-based UV Mappingcited by 9
  5. 2025RMAvatar: Photorealistic Human Avatar Reconstruction from Monocular Video Based on Rectified Mesh-embedded Gaussianscited by 7
  6. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  7. 2026BiMotion: B-spline Motion for Text-guided Dynamic 3D Character Generationcited by 4
  8. 2024AniArtAvatar: Animatable 3D Art Avatar from a Single Imagecited by 2
  9. 20254DGS-Craft: Consistent and Interactive 4D Gaussian Splatting Editingcited by 2
  10. 2025Dynamic-eDiTor: Training-Free Text-Driven 4D Scene Editing with Multimodal Diffusion Transformercited by 2
  11. 2026ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splattingcited by 2
  12. 2024EVLM: Self-Reflective Multimodal Reasoning for Cross-Dimensional Visual Editingcited by 1
  13. 2025Instruct-4DGS: Efficient Dynamic Scene Editing via 4D Gaussian-based Static-Dynamic Separationcited by 1
  14. 2025Advances in 4D Generation: A Surveycited by 1
  15. 2025MorphoSim: An Interactive, Controllable, and Editable Language-guided 4D World Simulatorcited by 1
  16. 2025Mono4DEditor: Text-Driven 4D Scene Editing from Monocular Video via Point-Level Localization of Language-Embedded Gaussianscited by 1
  17. 2025Smol-GS: Compact Representations for Abstract 3D Gaussian Splattingcited by 1
  18. 2026Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Modelscited by 1
  19. 2026Catalyst4D: High-Fidelity 3D-to-4D Scene Editing via Dynamic Propagationcited by 0
  20. 2026GSwap: Realistic Head Swapping with Dynamic Neural Gaussian Fieldcited by 0