# CAFA: a Controllable Automatic Foley Artist

arXiv: https://arxiv.org/abs/2504.06778 · HTML: https://mlsanity.com/paper/2504.06778/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Apr 2025.

## Built on by (most cited first, 5 of 5)

- [LTX-2: Efficient Joint Audio-Visual Foundation Model](https://mlsanity.com/paper/2601.03233/) (2026)
- [FoleyBench: A Benchmark For Video-to-Audio Models](https://mlsanity.com/paper/2511.13219/) (2025)
- [AVControl: Efficient Framework for Training Audio-Visual Controls](https://mlsanity.com/paper/2603.24793/) (2026)
- [Beyond Time Shifts: Adapting Omni-LLM as a Reference-Free Evaluator for Generative Audio-Visual Models](https://mlsanity.com/paper/2607.09091/) (2026)
- [CounterFlow: A Two-Phase Inference-Time Sampling for Counterfactual Video Foley Generation](https://mlsanity.com/paper/2605.18916/) (2026)

## Abstract (arXiv)

Foley is a key element in video production, refers to the process of adding an audio signal to a silent video while ensuring semantic and temporal alignment. In recent years, the rise of personalized content creation and advancements in automatic video-to-audio models have increased the demand for greater user control in the process. One possible approach is to incorporate text to guide audio generation. While supported by existing methods, challenges remain in ensuring compatibility between modalities, particularly when the text introduces additional information or contradicts the sounds naturally inferred from the visuals. In this work, we introduce CAFA (Controllable Automatic Foley Artist) a video-and-text-to-audio model that generates semantically and temporally aligned audio for a given video, guided by text input. CAFA is built upon a text-to-audio model and integrates video information through a modality adapter mechanism. By incorporating text, users can refine semantic details and introduce creative variations, guiding the audio synthesis beyond the expected video contextual cues. Experiments show that besides its superior quality in terms of semantic alignment and audio-visual synchronization the proposed method enable high textual controllability as demonstrated in subjective and objective evaluations.