# End2You -- The Imperial Toolkit for Multimodal Profiling by End-to-End Learning

arXiv: https://arxiv.org/abs/1802.01115 · HTML: https://mlsanity.com/paper/1802.01115/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2018.

## Built on by (most cited first, 12 of 12)

- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [The INTERSPEECH 2021 Computational Paralinguistics Challenge: COVID-19 Cough, COVID-19 Speech, Escalation & Primates](https://mlsanity.com/paper/2102.13468/) (2021)
- [The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress](https://mlsanity.com/paper/2207.05691/) (2022)
- [The ICML 2022 Expressive Vocalizations Workshop and Competition: Recognizing, Generating, and Personalizing Vocal Bursts](https://mlsanity.com/paper/2205.01780/) (2022)
- [DeepSpectrumLite: A Power-Efficient Transfer Learning Framework for Embedded Speech and Audio Processing from Decentralised Data](https://mlsanity.com/paper/2104.11629/) (2021)
- [Continuous-Time Audiovisual Fusion with Recurrence vs. Attention for In-The-Wild Affect Recognition](https://mlsanity.com/paper/2203.13285/) (2022)
- [The ACII 2022 Affective Vocal Bursts Workshop & Competition: Understanding a critically understudied modality of emotional expression](https://mlsanity.com/paper/2207.03572/) (2022)
- [Use of Affective Visual Information for Summarization of Human-Centric Videos](https://mlsanity.com/paper/2107.03783/) (2021)
- [A Summary of the ComParE COVID-19 Challenges](https://mlsanity.com/paper/2202.08981/) (2022)
- [Fine-tuning Wav2vec for Vocal-burst Emotion Recognition](https://mlsanity.com/paper/2210.00263/) (2022)
- [Facial Emotion Recognition using Deep Residual Networks in Real-World Environments](https://mlsanity.com/paper/2111.02717/) (2021)
- [Lightweight Models for Emotional Analysis in Video](https://mlsanity.com/paper/2503.10530/) (2025)

## Abstract (arXiv)

We introduce End2You -- the Imperial College London toolkit for multimodal profiling by end-to-end deep learning. End2You is an open-source toolkit implemented in Python and is based on Tensorflow. It provides capabilities to train and evaluate models in an end-to-end manner, i.e., using raw input. It supports input from raw audio, visual, physiological or other types of information or combination of those, and the output can be of an arbitrary representation, for either classification or regression tasks. To our knowledge, this is the first toolkit that provides generic end-to-end learning for profiling capabilities in either unimodal or multimodal cases. To test our toolkit, we utilise the RECOLA database as was used in the AVEC 2016 challenge. Experimental results indicate that End2You can provide comparable results to state-of-the-art methods despite no need of expert-alike feature representations, but self-learning these from the data "end to end".