Measuring Error Alignment for Decision-Making Systems
On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Abstract (arXiv)
Given that AI systems are set to play a pivotal role in future decision-making processes, their trustworthiness and reliability are of critical concern. Due to their scale and complexity, modern AI systems resist direct interpretation, and alternative ways are needed to establish trust in those systems, and determine how well they align with human values. We argue that good measures of the information processing similarities between AI and humans, may be able to achieve these same ends. While Representational alignment (RA) approaches measure similarity between the internal states of two systems, the associated data can be expensive and difficult to collect for human systems. In contrast, Behavioural alignment (BA) comparisons are cheaper and easier, but questions remain as to their sensitivity and reliability. We propose two new behavioural alignment metrics misclassification agreement which measures the similarity between the errors of two systems on the same instances, and class-level error similarity which measures the similarity between the error distributions of two systems. We show that our metrics correlate well with RA metrics, and provide complementary information to another BA metric, within a range of domains, and set the scene for a new approach to value alignment.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 41 of its 71 references that are in the map. First posted on arXiv 20 Sep 2024.
Builds on
41 of 71 references.
References · 41
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2014Going Deeper with Convolutionscited by 4,231
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2021ViViT: A Video Vision Transformercited by 1,610
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
29 more
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
- 2019Natural Adversarial Examplescited by 1,401
- 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
- 2016An overview of gradient descent optimization algorithmscited by 1,021
- 2021Scaling Vision Transformerscited by 993
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
- 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
- 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
- 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 483
- 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
- 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
- 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
- 2018Generalisation in humans and deep neural networkscited by 237
- 2019Human uncertainty makes classification more robustcited by 231
- 2021Partial success in closing the gap between human and machine visioncited by 185
- 2023Getting aligned on representational alignmentcited by 142
- 2020What shapes feature representations? Exploring datasets, architectures, and trainingcited by 125
- 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 82
- 2021Consequences of Misaligned AIcited by 79
- 2020Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistencycited by 77
- 2022Human alignment of neural network representationscited by 74
- 2015Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Datacited by 40
- 2021The Challenge of Value Alignment: from Fairer Algorithms to AI Safetycited by 34
- 2020Would Mega-scale Datasets Further Enhance Spatiotemporal 3D CNNs?cited by 31
- 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
- 2015Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognitioncited by 14
- 2016Humans and deep networks largely agree on which kinds of variation make object recognition hardercited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024A Flexible Method for Behaviorally Measuring Alignment Between Human and Artificial Intelligence Using Representational Similarity Analysis
- 2026Principles Do Not Apply Themselves: A Hermeneutic Perspective on AI Alignment
- 2024Position: Towards Bidirectional Human-AI Alignment
- 2024On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models
- 2024Challenges and Future Directions of Data-Centric AI Alignment
- 2025EigenBench: A Comparative Behavioral Measure of Value Alignment
- 2026Toward a Theory of Value in AI Alignment
- 2024ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs