SICI: A Semantic-Pragmatic Complexity Index Reveals Regime Shifts in LLM Stance Detection
Not on the map. Why?
Abstract (arXiv)
Prompt-based LLMs are increasingly used for stance detection, but harder examples are not always repaired by clearer instructions, reasoning prompts, retrieval, or debate. We introduce SICI (Stance Inference Complexity Index), a seven-dimensional diagnostic measure of the semantic-pragmatic burden imposed by a target--text pair. Across SemEval-2016 and VAST, SICI predicts LLM accuracy better than surface proxies and shows substantial cross-scorer reliability (α=0.771). More importantly, LLM errors change regime as SICI increases: low-complexity examples invite over-attribution, especially Against predictions; intermediate examples form an unstable boundary; and high-complexity examples rapidly concentrate on None. This phase-transition-like structure persists across GPT-3.5, GPT-4o-mini, DeepSeek-V3, and GPT-4o, although stronger models move the boundaries. A 15-method intervention study further shows that prompting, retrieval, and debate often shift models along the attribution--abstention axis rather than removing the high-complexity bottleneck.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 32 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
8 of 32 references.
References · 8
- 2020BERTweet: A pre-trained language model for English Tweetscited by 346
- 2020Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representationscited by 98
- 2020Stance Detection on Social Media: State of the Art and Trendscited by 95
- 2021Adversarial Learning for Zero-Shot Stance Detection on Social Mediacited by 37
- 2022Improved Target-specific Stance Detection on Social Media Platforms by Delving into Conversation Threadscited by 16
- 2023Chain-of-Thought Embeddings for Stance Detection on Social Mediacited by 13
- 2024Stance Reasoner: Zero-Shot Stance Detection on Social Media with Explicit Reasoningcited by 12
- 2025Are Stereotypes Leading LLMs' Zero-Shot Stance Detection ?cited by 1
Built on by
Not cited yet.