mlsanityData to 14 Oct 2026

CalliffusionV2: Personalized Natural Calligraphy Generation with Flexible Multi-modal Control

arXiv 2410.03787 ↗First posted 3 Oct 2024cs.CL cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce CalliffusionV2, a novel system designed to produce natural Chinese calligraphy with flexible multi-modal control. Unlike previous approaches that rely solely on image or text inputs and lack fine-grained control, our system leverages both images to guide generations at fine-grained levels and natural language texts to describe the features of generations. CalliffusionV2 excels at creating a broad range of characters and can quickly learn new styles through a few-shot learning approach. It is also capable of generating non-Chinese characters without prior training. Comprehensive tests confirm that our system produces calligraphy that is both stylistically accurate and recognizable by neural network classifiers and human evaluators.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 31 of its 33 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

31 of 33 references.

References · 31

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  10. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  11. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
19 more
  1. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  2. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  3. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  4. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
  5. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  6. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 244
  7. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  8. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuningcited by 208
  9. 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 196
  10. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 141
  11. 2021DG-Font: Deformable Generative Networks for Unsupervised Font Generationcited by 70
  12. 2023FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learningcited by 48
  13. 2020Few-shot Font Generation with Localized Style Representations and Factorizationcited by 47
  14. 2021Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Expertscited by 46
  15. 2020Few-shot Compositional Font Generation with Dual Memorycited by 44
  16. 2020CalliGAN: Style and Structure-aware Chinese Calligraphy Character Generatorcited by 28
  17. 2023CF-Font: Content Fusion for Few-shot Font Generationcited by 26
  18. 2023Calliffusion: Chinese Calligraphy Generation and Style Transfer with Diffusion Modelingcited by 9
  19. 2023CalliPaint: Chinese Calligraphy Inpainting with Diffusion Modelcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2025UniCalli: A Unified Diffusion Framework for Column-Level Generation and Recognition of Chinese Calligraphycited by 2
  2. 2024Moyun: A Diffusion-Based Model for Style-Specific Chinese Calligraphy Generationcited by 1
  3. 2026CalliMaster: Mastering Page-level Chinese Calligraphy via Layout-guided Spatial Planningcited by 0