# The BS-meter: Detecting Politics and Labour through ChatGPT's Language

arXiv: https://arxiv.org/abs/2411.15129 · HTML: https://mlsanity.com/paper/2411.15129/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 73 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Talking About Large Language Models](https://mlsanity.com/paper/2212.03551/) (2022)

## Abstract (arXiv)

What can we learn about language from studying how it is used by ChatGPT and other large language model (LLM)-based chatbots? In this paper, we analyse the distinctive character of language generated by ChatGPT, in relation to questions raised by natural language processing pioneer, and student of Wittgenstein, Margaret Masterman. Following frequent complaints that LLM-based chatbots produce "bullshit," in the sense of Frankfurt's popular monograph On Bullshit, we conduct an empirical study to contrast the language of 1,000 scientific publications with typical text generated by ChatGPT. We then explore whether the same language features can be detected in two well-known contexts of social dysfunction: George Orwell's critique of political speech, and David Graeber's characterisation of bullshit jobs. Using simple hypothesis-testing methods, we demonstrate that a statistical model of bullshit can reliably relate the Frankfurtian artificial bullshit of ChatGPT to the political and workplace functions of bullshit as observed in natural human language.