# Comparing PCG metrics with Human Evaluation in Minecraft Settlement Generation

arXiv: https://arxiv.org/abs/2107.02457 · HTML: https://mlsanity.com/paper/2107.02457/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 43 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Procedural Content Generation via Machine Learning (PCGML)](https://mlsanity.com/paper/1702.00539/) (2017)
- [The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors](https://mlsanity.com/paper/2101.11071/) (2021)
- [Generative Design in Minecraft (GDMC), Settlement Generation Competition](https://mlsanity.com/paper/1803.09853/) (2018)
- [General Video Game Rule Generation](https://mlsanity.com/paper/1906.05160/) (2019)
- [The AI Settlement Generation Challenge in Minecraft: First Year Report](https://mlsanity.com/paper/2103.14950/) (2021)
- [Generative Design in Minecraft: Chronicle Challenge](https://mlsanity.com/paper/1905.05888/) (2019)
- [A Continuous Information Gain Measure to Find the Most Discriminatory Problems for AI Benchmarking](https://mlsanity.com/paper/1809.02904/) (2018)
- [Demonstrating the Feasibility of Automatic Game Balancing](https://mlsanity.com/paper/1603.03795/) (2016)
- [Exploring Gameplay With AI Agents](https://mlsanity.com/paper/1811.06962/) (2018)

## Built on by (most cited first, 5 of 5)

- [Measuring Diversity of Game Scenarios](https://mlsanity.com/paper/2404.15192/) (2024)
- [Automated Isovist Computation for Minecraft](https://mlsanity.com/paper/2204.03752/) (2022)
- [Exploring Minecraft Settlement Generators with Generative Shift Analysis](https://mlsanity.com/paper/2309.05371/) (2023)
- [Believable Minecraft Settlements by Means of Decentralised Iterative Planning](https://mlsanity.com/paper/2309.10871/) (2023)
- [Landmarks, Monuments, and Beacons: Understanding Generative Calls to Action](https://mlsanity.com/paper/2509.19030/) (2025)

## Abstract (arXiv)

There are a range of metrics that can be applied to the artifacts produced by procedural content generation, and several of them come with qualitative claims. In this paper, we adapt a range of existing PCG metrics to generated Minecraft settlements, develop a few new metrics inspired by PCG literature, and compare the resulting measurements to existing human evaluations. The aim is to analyze how those metrics capture human evaluation scores in different categories, how the metrics generalize to another game domain, and how metrics deal with more complex artifacts. We provide an exploratory look at a variety of metrics and provide an information gain and several correlation analyses. We found some relationships between human scores and metrics counting specific elements, measuring the diversity of blocks and measuring the presence of crafting materials for the present complex blocks.