# Sentiment Reasoning for Healthcare

arXiv: https://arxiv.org/abs/2407.21054 · HTML: https://mlsanity.com/paper/2407.21054/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 25 of its 117 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1904.02232/) (2019)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generation](https://mlsanity.com/paper/2205.06457/) (2022)
- [BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese](https://mlsanity.com/paper/2109.09701/) (2021)
- [Crosslingual Generalization through Multitask Finetuning](https://mlsanity.com/paper/2211.01786/) (2022)
- [Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey](https://mlsanity.com/paper/2007.01127/) (2020)
- [Instruction Tuning for Few-Shot Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2210.06629/) (2022)
- [Investigating Mysteries of CoT-Augmented Distillation](https://mlsanity.com/paper/2406.14511/) (2024)
- [Distilling Reasoning Ability from Large Language Models with Adaptive Thinking](https://mlsanity.com/paper/2404.09170/) (2024)
- [wav2vec: Unsupervised Pre-training for Speech Recognition](https://mlsanity.com/paper/1904.05862/) (2019)
- [UniSpeech at scale: An Empirical Study of Pre-training Method on Large-Scale Speech Recognition Dataset](https://mlsanity.com/paper/2107.05233/) (2021)
- [Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors](https://mlsanity.com/paper/2203.00257/) (2022)

## Abstract (arXiv)

Transparency in AI healthcare decision-making is crucial. By incorporating rationales to explain reason for each predicted label, users could understand Large Language Models (LLMs)'s reasoning to make better decision. In this work, we introduce a new task - Sentiment Reasoning - for both speech and text modalities, and our proposed multimodal multitask framework and the world's largest multimodal sentiment analysis dataset. Sentiment Reasoning is an auxiliary task in sentiment analysis where the model predicts both the sentiment label and generates the rationale behind it based on the input transcript. Our study conducted on both human transcripts and Automatic Speech Recognition (ASR) transcripts shows that Sentiment Reasoning helps improve model transparency by providing rationale for model prediction with quality semantically comparable to humans while also improving model's classification performance (+2% increase in both accuracy and macro-F1) via rationale-augmented fine-tuning. Also, no significant difference in the semantic quality of generated rationales between human and ASR transcripts. All code, data (five languages - Vietnamese, English, Chinese, German, and French) and models are published online: https://github.com/leduckhai/Sentiment-Reasoning