Affective social anthropomorphic intelligent system
Not on the map. Why?
Abstract (arXiv)
Human conversational styles are measured by the sense of humor, personality, and tone of voice. These characteristics have become essential for conversational intelligent virtual assistants. However, most of the state-of-the-art intelligent virtual assistants (IVAs) are failed to interpret the affective semantics of human voices. This research proposes an anthropomorphic intelligent system that can hold a proper human-like conversation with emotion and personality. A voice style transfer method is also proposed to map the attributes of a specific emotion. Initially, the frequency domain data (Mel-Spectrogram) is created by converting the temporal audio wave data, which comprises discrete patterns for audio features such as notes, pitch, rhythm, and melody. A collateral CNN-Transformer-Encoder is used to predict seven different affective states from voice. The voice is also fed parallelly to the deep-speech, an RNN model that generates the text transcription from the spectrogram. Then the transcripted text is transferred to the multi-domain conversation agent using blended skill talk, transformer-based retrieve-and-generate generation strategy, and beam-search decoding, and an appropriate textual response is generated. The system learns an invertible mapping of data to a latent space that can be manipulated and generates a Mel-spectrogram frame based on previous Mel-spectrogram frames to voice synthesize and style transfer. Finally, the waveform is generated using WaveGlow from the spectrogram. The outcomes of the studies we conducted on individual models were auspicious. Furthermore, users who interacted with the system provided positive feedback, demonstrating the system's effectiveness.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 31 of its 72 references that are in the map. First posted on arXiv 19 Apr 2023.
Builds on
31 of 72 references.
References · 31
- 2017Attention Is All You Needcited by 60,361
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2022Generative Adversarial Networkscited by 5,627
- 2014Going Deeper with Convolutionscited by 4,241
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2016Density estimation using Real NVPcited by 2,042
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
19 more
- 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
- 2020Recipes for building an open-domain chatbotcited by 593
- 2017Generative Adversarial Networks: An Overviewcited by 590
- 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
- 2020Towards a Human-like Open-Domain Chatbotcited by 470
- 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 431
- 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
- 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 309
- 2016A Persona-Based Neural Conversation Modelcited by 223
- 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 166
- 2020Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skillscited by 131
- 2020Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesiscited by 54
- 2015Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networkscited by 46
- 2017Emotional End-to-End Neural Speech Synthesizercited by 30
- 2018Expressive Speech Synthesis via Modeling Expressions with Variational Autoencodercited by 22
- 2018Voice Impersonation using Generative Adversarial Networkscited by 17
- 2017Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networkscited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.