mlsanityData to 14 Oct 2026

CalliRewrite: Recovering Handwriting Behaviors from Calligraphy Images without Supervision

arXiv 2405.15776 ↗First posted 20 Mar 2024cs.RO cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Human-like planning skills and dexterous manipulation have long posed challenges in the fields of robotics and artificial intelligence (AI). The task of reinterpreting calligraphy presents a formidable challenge, as it involves the decomposition of strokes and dexterous utensil control. Previous efforts have primarily focused on supervised learning of a single instrument, limiting the performance of robots in the realm of cross-domain text replication. To address these challenges, we propose CalliRewrite: a coarse-to-fine approach for robot arms to discover and recover plausible writing orders from diverse calligraphy images without requiring labeled demonstrations. Our model achieves fine-grained control of various writing utensils. Specifically, an unsupervised image-to-sequence model decomposes a given calligraphy glyph to obtain a coarse stroke sequence. Using an RL algorithm, a simulated brush is fine-tuned to generate stylized trajectories for robotic arm control. Evaluation in simulation and physical robot scenarios reveals that our method successfully replicates unseen fonts and styles while achieving integrity in unknown characters.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 19 of its 49 references that are in the map. First posted on arXiv 20 Mar 2024.

Builds on

19 of 49 references.

References · 19

  1. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  2. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  3. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  4. 2017A Neural Representation of Sketch Drawingscited by 313
  5. 2019Learning to Paint With Model-based Deep Reinforcement Learningcited by 71
  6. 2021DG-Font: Deformable Generative Networks for Unsupervised Font Generationcited by 70
  7. 2017Auto-Encoder Guided GAN for Chinese Calligraphy Synthesiscited by 29
  8. 2022Learning to Use Chopsticks in Diverse Gripping Stylescited by 29
  9. 2023DeepVecFont-v2: Exploiting Transformers to Synthesize Vector Fonts with Higher Qualitycited by 27
  10. 2020Content Masked Loss: Human-Like Brush Stroke Planning in a Reinforcement Learning Painting Agentcited by 15
  11. 2020PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planningcited by 12
  12. 2018Pyramid Embedded Generative Adversarial Network for Automated Font Generationcited by 11
7 more
  1. 2023VecFontSDF: Learning to Reconstruct and Synthesize High-quality Vector Fonts via Signed Distance Functionscited by 7
  2. 2020Artistic Style in Robotic Painting; a Machine Learning Approach to Learning Brushstroke from Human Artistscited by 6
  3. 2022FontTransformer: Few-shot High-resolution Chinese Glyph Image Synthesis via Stacked Transformerscited by 6
  4. 2020Robot Calligraphy using Pseudospectral Optimal Control in Conjunction with a Novel Dynamic Brush Modelcited by 4
  5. 2022Robot Navigation with Reinforcement Learned Path Generation and Fine-Tuned Motion Controlcited by 2
  6. 2021CLAMGen: Closed-Loop Arm Motion Generation via Multi-view Vision-Based RLcited by 1
  7. 2023End-to-end Manipulator Calligraphy Planning via Variational Imitation Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Bridging Online and Offline Handwriting via Differentiable Physical Renderingcited by 0