# The Touch'e23-ValueEval Dataset for Identifying Human Values behind Arguments

arXiv: https://arxiv.org/abs/2301.13771 · HTML: https://mlsanity.com/paper/2301.13771/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 30 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 6 of 6)

- [Social Chemistry 101: Learning to Reason about Social and Moral Norms](https://mlsanity.com/paper/2011.00620/) (2020)
- [ParsBERT: Transformer-based Model for Persian Language Understanding](https://mlsanity.com/paper/2005.12515/) (2020)
- [A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis](https://mlsanity.com/paper/1911.11408/) (2019)
- [ValueNet: A New Dataset for Human Value Driven Dialogue System](https://mlsanity.com/paper/2112.06346/) (2021)
- [Aligning to Social Norms and Values in Interactive Narratives](https://mlsanity.com/paper/2205.01975/) (2022)
- [Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits](https://mlsanity.com/paper/2301.00355/) (2023)

## Built on by (most cited first, 28 of 28)

- [The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values](https://mlsanity.com/paper/2310.07629/) (2023)
- [Value Portrait: Assessing Language Models' Values through Psychometrically and Ecologically Valid Items](https://mlsanity.com/paper/2505.01015/) (2025)
- [Measuring Human and AI Values Based on Generative Psychometrics with Large Language Models](https://mlsanity.com/paper/2409.12106/) (2024)
- [From Values to Opinions: Predicting Human Behaviors and Stances Using Value-Injected Large Language Models](https://mlsanity.com/paper/2310.17857/) (2023)
- [Unintended Harms of Value-Aligned LLMs: Psychological and Empirical Insights](https://mlsanity.com/paper/2506.06404/) (2025)
- [Adam-Smith at SemEval-2023 Task 4: Discovering Human Values in Arguments with Ensembles of Transformer-based Models](https://mlsanity.com/paper/2305.08625/) (2023)
- [Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models](https://mlsanity.com/paper/2510.18526/) (2025)
- [Epicurus at SemEval-2023 Task 4: Improving Prediction of Human Values behind Arguments by Leveraging Their Definitions](https://mlsanity.com/paper/2302.13925/) (2023)
- [MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions](https://mlsanity.com/paper/2403.07910/) (2024)
- [AdAEM: An Adaptively and Automated Extensible Measurement of LLMs' Value Difference](https://mlsanity.com/paper/2505.13531/) (2025)
- [Will Annotators Disagree? Identifying Subjectivity in Value-Laden Arguments](https://mlsanity.com/paper/2509.06704/) (2025)
- [Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum](https://mlsanity.com/paper/2601.14172/) (2026)
- [Do Schwartz Higher-Order Values Help Sentence-Level Human Value Detection? A Study of Hierarchical Gating and Calibration](https://mlsanity.com/paper/2602.00913/) (2026)
- [Rudolf Christoph Eucken at SemEval-2023 Task 4: An Ensemble Approach for Identifying Human Values from Arguments](https://mlsanity.com/paper/2305.05335/) (2023)
- [Consolidating Strategies for Countering Hate Speech Using Persuasive Dialogues](https://mlsanity.com/paper/2401.07810/) (2024)
- [From Argumentation to Deliberation: Perspectivized Stance Vectors for Fine-grained (Dis)agreement Analysis](https://mlsanity.com/paper/2502.09644/) (2025)
- [Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer](https://mlsanity.com/paper/2606.11018/) (2026)
- [Detecting value-expressive text posts in Russian social media](https://mlsanity.com/paper/2312.08968/) (2023)
- [Explainable Moral Values: a neuro-symbolic approach to value classification](https://mlsanity.com/paper/2410.12631/) (2024)
- [Growth First, Care Second? Tracing the Landscape of LLM Value Preferences in Everyday Dilemmas](https://mlsanity.com/paper/2602.04456/) (2026)
- [More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts](https://mlsanity.com/paper/2605.22641/) (2026)
- [KVpop -- Key-Value Cache Compression with Predictive Online Pruning](https://mlsanity.com/paper/2607.05061/) (2026)
- [Fantastic Biases (What are They) and Where to Find Them](https://mlsanity.com/paper/2411.15051/) (2024)
- [Prompt-Based Value Steering of Large Language Models](https://mlsanity.com/paper/2511.16688/) (2025)
- [Event-Centric Human Value Understanding in News-Domain Texts: An Actor-Conditioned, Multi-Granularity Benchmark](https://mlsanity.com/paper/2603.17838/) (2026)
- [Which Values Do LLMs Confuse? A Schwartz-Based Recognition Study](https://mlsanity.com/paper/2607.20270/) (2026)
- [Steering Geometry: Validating Human Value Geometry in LLM Steering Space](https://mlsanity.com/paper/2609.06289/) (2026)
- [Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations](https://mlsanity.com/paper/2610.19817/) (2026)

## Abstract (arXiv)

We present the Touch'e23-ValueEval Dataset for Identifying Human Values behind Arguments. To investigate approaches for the automated detection of human values behind arguments, we collected 9324 arguments from 6 diverse sources, covering religious texts, political discussions, free-text arguments, newspaper editorials, and online democracy platforms. Each argument was annotated by 3 crowdworkers for 54 values. The Touch'e23-ValueEval dataset extends the Webis-ArgValues-22. In comparison to the previous dataset, the effectiveness of a 1-Baseline decreases, but that of an out-of-the-box BERT model increases. Therefore, though the classification difficulty increased as per the label distribution, the larger dataset allows for training better models.