mlsanityData to 13 Oct 2026

Affective Computing Has Changed: The Foundation Model Disruption

arXiv 2409.08907 ↗First posted 13 Sep 2024cs.AI cs.CL cs.CYcited by 7

Not on the map. Why?

Abstract (arXiv)

The dawn of Foundation Models has on the one hand revolutionised a wide range of research problems, and, on the other hand, democratised the access and use of AI-based tools by the general public. We even observe an incursion of these models into disciplines related to human psychology, such as the Affective Computing domain, suggesting their affective, emerging capabilities. In this work, we aim to raise awareness of the power of Foundation Models in the field of Affective Computing by synthetically generating and analysing multimodal affective data, focusing on vision, linguistics, and speech (acoustics). We also discuss some fundamental problems, such as ethical issues and regulatory aspects, related to the use of Foundation Models in this research area.

Key facts. Cited by 7 papers in the map (data through 13 Oct 2026). Builds on 51 of its 85 references that are in the map. First posted on arXiv 13 Sep 2024.

Builds on

51 of 85 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,106
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  6. 2023GPT-4 Technical Reportcited by 18,278
  7. 2022Training language models to follow instructions with human feedbackcited by 15,856
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  11. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,492
  12. 2023Visual Instruction Tuningcited by 7,827
38 more
  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,564
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,389
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,323
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,284
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,585
  6. 2023Mistral 7Bcited by 3,123
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,044
  8. 2024Mixtral of Expertscited by 1,811
  9. 2016Semantics derived automatically from language corpora contain human-like biasescited by 895
  10. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 745
  11. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 601
  12. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 520
  13. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 491
  14. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 481
  15. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  16. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 278
  17. 2018Deep Facial Expression Recognition: A Surveycited by 270
  18. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  19. 2023Multimodal Large Language Models: A Surveycited by 194
  20. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 145
  21. 2023X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languagescited by 128
  22. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  23. 2023AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Modelcited by 93
  24. 2023X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoningcited by 79
  25. 2020Towards causal benchmarking of bias in face analysis algorithmscited by 64
  26. 2019SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wildcited by 62
  27. 2022A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advancescited by 59
  28. 2023Emotional Intelligence of Large Language Modelscited by 58
  29. 2017Affect-LM: A Neural Language Model for Customizable Affective Text Generationcited by 50
  30. 2017ExprGAN: Facial Expression Editing with Controllable Expression Intensitycited by 47
  31. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 47
  32. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 32
  33. 2023A Wide Evaluation of ChatGPT on Affective Computing Taskscited by 30
  34. 2022Speech Synthesis with Mixed Emotionscited by 28
  35. 2023Fine-grained Affective Processing Capabilities Emerging from Large Language Modelscited by 24
  36. 2024ModaVerse: Efficiently Transforming Modalities with LLMscited by 18
  37. 2020Facial Expressions as a Vulnerability in Face Recognitioncited by 11
  38. 2024Computer Audition: From Task-Specific Machine Learning to Foundation Modelscited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 4.

7 papers.

  1. 2025Large Language Models for Depression Recognition in Spoken Language Integrating Psychological Knowledgecited by 3
  2. 2024Towards Friendly AI: A Comprehensive Review and New Perspectives on Human-AI Alignmentcited by 2
  3. 2025Reading Smiles: Proxy Bias in Foundation Models for Facial Emotion Recognitioncited by 1
  4. 2026A Pilot Study on Curator-Guided Multilingual Art Description for Blind and Low-Vision Audiences with Small Vision-Language Modelscited by 0
  5. 2026Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognitioncited by 0
  6. 2026Retrieval-Augmented Personalization with Foundation Models for Wearable Stress Detectioncited by 0
  7. 2026Spook the Machine: Gamified Exploration of Human Imagination of Machine Fearcited by 0