# CultureBERT: Measuring Corporate Culture With Transformer-Based Language Models

arXiv: https://arxiv.org/abs/2212.00509 · HTML: https://mlsanity.com/paper/2212.00509/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 44 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [FinBERT: Financial Sentiment Analysis with Pre-trained Language Models](https://mlsanity.com/paper/1908.10063/) (2019)
- [Comparing BERT against traditional machine learning text classification](https://mlsanity.com/paper/2005.13012/) (2020)

## Built on by (most cited first, 3 of 3)

- [LLM Content Moderation and User Satisfaction: Evidence from Response Refusals in Chatbot Arena](https://mlsanity.com/paper/2501.03266/) (2025)
- [Human-AI Interaction and User Satisfaction: Empirical Evidence from Online Reviews of AI Products](https://mlsanity.com/paper/2503.17955/) (2025)
- [Multilingual De-Duplication Strategies: Applying scalable similarity search with monolingual & multilingual embedding models](https://mlsanity.com/paper/2406.13695/) (2024)

## Abstract (arXiv)

This paper introduces transformer-based language models to the literature measuring corporate culture from text documents. We compile a unique data set of employee reviews that were labeled by human evaluators with respect to the information the reviews reveal about the firms' corporate culture. Using this data set, we fine-tune state-of-the-art transformer-based language models to perform the same classification task. In out-of-sample predictions, our language models classify 17 to 30 percentage points more of employee reviews in line with human evaluators than traditional approaches of text classification. We make our models publicly available.