# A Fractal Approach to Characterize Emotions in Audio and Visual Domain: A Study on Cross-Modal Interaction

arXiv: https://arxiv.org/abs/2102.06038 · HTML: https://mlsanity.com/paper/2102.06038/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 8 of 8)

- [Emotion Specification from Musical Stimuli: An EEG Study with AFA and DFA](https://mlsanity.com/paper/1705.00228/) (2017)
- [Can Musical Emotion Be Quantified With Neural Jitter Or Shimmer? A Novel EEG Based Study With Hindustani Classical Music](https://mlsanity.com/paper/1705.03543/) (2017)
- [Music of Brain and Music on Brain: A Novel EEG Sonification approach](https://mlsanity.com/paper/1712.08336/) (2017)
- [A Non Linear Approach towards Automated Emotion Analysis in Hindustani Music](https://mlsanity.com/paper/1612.00172/) (2016)
- [Gestalt Phenomenon in Music? A Neurocognitive Physics Study with EEG](https://mlsanity.com/paper/1703.06491/) (2017)
- [Neural (EEG) Response during Creation and Appreciation: A Novel Study with Hindustani Raga Music](https://mlsanity.com/paper/1704.05687/) (2017)
- [Speaker Recognition in Bengali Language from Nonlinear Features](https://mlsanity.com/paper/2004.07820/) (2020)
- [Acoustical classification of different speech acts using nonlinear methods](https://mlsanity.com/paper/2004.08248/) (2020)

## Abstract (arXiv)

It is already known that both auditory and visual stimulus is able to convey emotions in human mind to different extent. The strength or intensity of the emotional arousal vary depending on the type of stimulus chosen. In this study, we try to investigate the emotional arousal in a cross-modal scenario involving both auditory and visual stimulus while studying their source characteristics. A robust fractal analytic technique called Detrended Fluctuation Analysis (DFA) and its 2D analogue has been used to characterize three (3) standardized audio and video signals quantifying their scaling exponent corresponding to positive and negative valence. It was found that there is significant difference in scaling exponents corresponding to the two different modalities. Detrended Cross Correlation Analysis (DCCA) has also been applied to decipher degree of cross-correlation among the individual audio and visual stimulus. This is the first of its kind study which proposes a novel algorithm with which emotional arousal can be classified in cross-modal scenario using only the source audio and visual signals while also attempting a correlation between them.