# Automatic Stage Lighting Control: Is it a Rule-Driven Process or Generative Task?

arXiv: https://arxiv.org/abs/2506.01482 · HTML: https://mlsanity.com/paper/2506.01482/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 2 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch](https://mlsanity.com/paper/2311.03099/) (2023)
- [Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers](https://mlsanity.com/paper/2409.20537/) (2024)
- [UniAudio: An Audio Foundation Model Toward Universal Audio Generation](https://mlsanity.com/paper/2310.00704/) (2023)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training](https://mlsanity.com/paper/2106.05630/) (2021)
- [LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT](https://mlsanity.com/paper/2310.04673/) (2023)
- [YuE: Scaling Open Foundation Models for Long-Form Music Generation](https://mlsanity.com/paper/2503.08638/) (2025)
- [VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling](https://mlsanity.com/paper/2406.04321/) (2024)
- [Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model](https://mlsanity.com/paper/2311.00968/) (2023)
- [Comparison and Analysis of Deep Audio Embeddings for Music Emotion Recognition](https://mlsanity.com/paper/2104.06517/) (2021)
- [PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training](https://mlsanity.com/paper/2407.03361/) (2024)
- [CrossFi: A Cross Domain Wi-Fi Sensing Framework Based on Siamese Network](https://mlsanity.com/paper/2408.10919/) (2024)
- [Let Network Decide What to Learn: Symbolic Music Understanding Model Based on Large-scale Adversarial Pre-training](https://mlsanity.com/paper/2407.08306/) (2024)
- [BEAST: Online Joint Beat and Downbeat Tracking Based on Streaming Transformer](https://mlsanity.com/paper/2312.17156/) (2023)
- [MV-Crafter: An Intelligent System for Music-guided Video Generation](https://mlsanity.com/paper/2504.17267/) (2025)

## Built on by (most cited first, 4 of 4)

- [Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoring](https://mlsanity.com/paper/2507.17442/) (2025)
- [Bridging MARL to SARL: An Order-Independent Multi-Agent Transformer via Latent Consensus](https://mlsanity.com/paper/2604.13472/) (2026)
- [Zero-Effort Image-to-Music Generation: An Interpretable RAG-based VLM Approach](https://mlsanity.com/paper/2509.22378/) (2025)
- [Stage Light is Sequence2: Multi-Light Control via Imitation Learning](https://mlsanity.com/paper/2605.03660/) (2026)

## Abstract (arXiv)

Stage lighting is a vital component in live music performances, shaping an engaging experience for both musicians and audiences. In recent years, Automatic Stage Lighting Control (ASLC) has attracted growing interest due to the high costs of hiring or training professional lighting engineers. However, most existing ASLC solutions only classify music into limited categories and map them to predefined light patterns, resulting in formulaic and monotonous outcomes that lack rationality. To address this gap, this paper presents Skip-BART, an end-to-end model that directly learns from experienced lighting engineers and predict vivid, human-like stage lighting. To the best of our knowledge, this is the first work to conceptualize ASLC as a generative task rather than merely a classification problem. Our method adapts the BART model to take audio music as input and produce light hue and value (intensity) as output, incorporating a novel skip connection mechanism to enhance the relationship between music and light within the frame grid. To address the lack of available datasets, we create the first stage lighting dataset, along with several pre-training and transfer learning techniques to improve model training with limited data. We validate our method through both quantitative analysis and an human evaluation, demonstrating that Skip-BART outperforms conventional rule-based methods across all evaluation metrics and shows only a limited gap compared to real lighting engineers. To support further research, we have made our self-collected dataset, code, and trained model parameters available at https://github.com/RS2002/Skip-BART .