# SubjQA: A Dataset for Subjectivity and Review Comprehension

arXiv: https://arxiv.org/abs/2004.14283 · HTML: https://mlsanity.com/paper/2004.14283/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2020.

## Built on by (most cited first, 24 of 24)

- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension](https://mlsanity.com/paper/2107.12708/) (2021)
- [Context-faithful Prompting for Large Language Models](https://mlsanity.com/paper/2303.11315/) (2023)
- [Constrained Optimization with Dynamic Bound-scaling for Effective NLPBackdoor Defense](https://mlsanity.com/paper/2202.05749/) (2022)
- [Generative Language Models for Paragraph-Level Question Generation](https://mlsanity.com/paper/2210.03992/) (2022)
- [English Machine Reading Comprehension Datasets: A Survey](https://mlsanity.com/paper/2101.10421/) (2021)
- [SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding](https://mlsanity.com/paper/2308.10529/) (2023)
- [Mirror: A Universal Framework for Various Information Extraction Tasks](https://mlsanity.com/paper/2311.05419/) (2023)
- [AdvisorQA: Towards Helpful and Harmless Advice-seeking Question Answering with Collective Intelligence](https://mlsanity.com/paper/2404.11826/) (2024)
- [DebateQA: Evaluating Question Answering on Debatable Knowledge](https://mlsanity.com/paper/2408.01419/) (2024)
- [To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO](https://mlsanity.com/paper/2404.04575/) (2024)
- [Product Question Answering in E-Commerce: A Survey](https://mlsanity.com/paper/2302.08092/) (2023)
- [A Practical Toolkit for Multilingual Question and Answer Generation](https://mlsanity.com/paper/2305.17416/) (2023)
- ["What do others think?": Task-Oriented Conversational Modeling with Subjective Knowledge](https://mlsanity.com/paper/2305.12091/) (2023)
- [BSharedRAG: Backbone Shared Retrieval-Augmented Generation for the E-commerce Domain](https://mlsanity.com/paper/2409.20075/) (2024)
- [Explicit Diversity Conditions for Effective Question Answer Generation with Large Language Models](https://mlsanity.com/paper/2406.17990/) (2024)
- [NeuroQL: A Neuro-Symbolic Language and Dataset for Inter-Subjective Reasoning](https://mlsanity.com/paper/2303.07146/) (2023)
- [Acoustic-based Gender Differentiation in Speech-aware Language Models](https://mlsanity.com/paper/2509.21125/) (2025)
- [OpinionConv: Conversational Product Search with Grounded Opinions](https://mlsanity.com/paper/2308.04226/) (2023)
- [Synthetic Dataset Creation and Fine-Tuning of Transformer Models for Question Answering in Serbian](https://mlsanity.com/paper/2404.08617/) (2024)
- [Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering](https://mlsanity.com/paper/2409.16025/) (2024)
- [A Dataset for Spatiotemporal-Sensitive POI Question Answering](https://mlsanity.com/paper/2505.10928/) (2025)
- [Do We Still Need Humans in the Loop? Human vs. LLM Annotation in Active Learning for TikTok Hate Speech Detection](https://mlsanity.com/paper/2604.13899/) (2026)

## Abstract (arXiv)

Subjectivity is the expression of internal opinions or beliefs which cannot be objectively observed or verified, and has been shown to be important for sentiment analysis and word-sense disambiguation. Furthermore, subjectivity is an important aspect of user-generated data. In spite of this, subjectivity has not been investigated in contexts where such data is widespread, such as in question answering (QA). We therefore investigate the relationship between subjectivity and QA, while developing a new dataset. We compare and contrast with analyses from previous work, and verify that findings regarding subjectivity still hold when using recently developed NLP architectures. We find that subjectivity is also an important feature in the case of QA, albeit with more intricate interactions between subjectivity and QA performance. For instance, a subjective question may or may not be associated with a subjective answer. We release an English QA dataset (SubjQA) based on customer reviews, containing subjectivity annotations for questions and answer spans across 6 distinct domains.