# CustomTTT: Motion and Appearance Customized Video Generation via Test-Time Training

arXiv: https://arxiv.org/abs/2412.15646 · HTML: https://mlsanity.com/paper/2412.15646/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 20 Dec 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [TIES-Merging: Resolving Interference When Merging Models](https://mlsanity.com/paper/2306.01708/) (2023)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Test-Time Training with Masked Autoencoders](https://mlsanity.com/paper/2209.07522/) (2022)
- [DreamVideo: Composing Your Dream Videos with Customized Subject and Motion](https://mlsanity.com/paper/2312.04433/) (2023)
- [MotionDirector: Motion Customization of Text-to-Video Diffusion Models](https://mlsanity.com/paper/2310.08465/) (2023)
- [Tent: Fully Test-time Adaptation by Entropy Minimization](https://mlsanity.com/paper/2006.10726/) (2020)
- [Inserting Anybody in Diffusion Models via Celeb Basis](https://mlsanity.com/paper/2306.00926/) (2023)
- [Test-Time Training with Self-Supervision for Generalization under Distribution Shifts](https://mlsanity.com/paper/1909.13231/) (2019)
- [Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models](https://mlsanity.com/paper/2402.14780/) (2024)
- [Test-Time Training on Video Streams](https://mlsanity.com/paper/2307.05014/) (2023)
- [Motion Inversion for Video Customization](https://mlsanity.com/paper/2403.20193/) (2024)

## Built on by (most cited first, 9 of 9)

- [Cross-Frame Representation Alignment for Fine-Tuning Video Diffusion Models](https://mlsanity.com/paper/2506.09229/) (2025)
- [SMRABooth: Subject and Motion Representation Alignment for Customized Video Generation](https://mlsanity.com/paper/2512.12193/) (2025)
- [SynMotion: Semantic-Visual Adaptation for Motion Customized Video Generation](https://mlsanity.com/paper/2506.23690/) (2025)
- [Lay2Story: Extending Diffusion Transformers for Layout-Togglable Story Generation](https://mlsanity.com/paper/2508.08949/) (2025)
- [InnoAds-Composer: Efficient Condition Composition for E-Commerce Poster Generation](https://mlsanity.com/paper/2603.05898/) (2026)
- [JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generation](https://mlsanity.com/paper/2503.23951/) (2025)
- [In-Context Sync-LoRA for Portrait Video Editing](https://mlsanity.com/paper/2512.03013/) (2025)
- [ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing](https://mlsanity.com/paper/2607.21529/) (2026)
- [Composing Concepts from Images and Videos via Concept-prompt Binding](https://mlsanity.com/paper/2512.09824/) (2025)

## Abstract (arXiv)

Benefiting from large-scale pre-training of text-video pairs, current text-to-video (T2V) diffusion models can generate high-quality videos from the text description. Besides, given some reference images or videos, the parameter-efficient fine-tuning method, i.e. LoRA, can generate high-quality customized concepts, e.g., the specific subject or the motions from a reference video. However, combining the trained multiple concepts from different references into a single network shows obvious artifacts. To this end, we propose CustomTTT, where we can joint custom the appearance and the motion of the given video easily. In detail, we first analyze the prompt influence in the current video diffusion model and find the LoRAs are only needed for the specific layers for appearance and motion customization. Besides, since each LoRA is trained individually, we propose a novel test-time training technique to update parameters after combination utilizing the trained customized models. We conduct detailed experiments to verify the effectiveness of the proposed methods. Our method outperforms several state-of-the-art works in both qualitative and quantitative evaluations.