# Automotive Perception Software Development: An Empirical Investigation into Data, Annotation, and Ecosystem Challenges

arXiv: https://arxiv.org/abs/2303.05947 · HTML: https://mlsanity.com/paper/2303.05947/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 56 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure](https://mlsanity.com/paper/2010.13561/) (2020)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)
- [From ImageNet to Image Classification: Contextualizing Progress on Benchmarks](https://mlsanity.com/paper/2005.11295/) (2020)
- [Requirements Engineering for Machine Learning: Perspectives from Data Scientists](https://mlsanity.com/paper/1908.04674/) (2019)
- [Is one annotation enough? A data-centric image classification benchmark for noisy and ambiguous label estimation](https://mlsanity.com/paper/2207.06214/) (2022)
- [Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry](https://mlsanity.com/paper/1812.05389/) (2018)
- [Whose AI Dream? In search of the aspiration in data annotation](https://mlsanity.com/paper/2203.10748/) (2022)
- [Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262](https://mlsanity.com/paper/1808.01614/) (2018)
- [If a Human Can See It, So Should Your System: Reliability Requirements for Machine Vision Components](https://mlsanity.com/paper/2202.03930/) (2022)
- [Impact of Ground Truth Annotation Quality on Performance of Semantic Image Segmentation of Traffic Conditions](https://mlsanity.com/paper/1901.00001/) (2018)
- [Thematic Fit Bits: Annotation Quality and Quantity Interplay for Event Participant Representation](https://mlsanity.com/paper/2105.06097/) (2021)
- [Painting the Landscape of Automotive Software in GitHub](https://mlsanity.com/paper/2203.08936/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [AutoVDC: Automated Vision Data Cleaning Using Vision-Language Models](https://mlsanity.com/paper/2507.12414/) (2025)
- [LLM-Powered Workflow Optimization for Multidisciplinary Software Development: An Automotive Industry Case Study](https://mlsanity.com/paper/2603.21439/) (2026)
- [Data Leakage in Automotive Perception: Practitioners' Insights](https://mlsanity.com/paper/2604.06899/) (2026)

## Abstract (arXiv)

Software that contains machine learning algorithms is an integral part of automotive perception, for example, in driving automation systems. The development of such software, specifically the training and validation of the machine learning components, require large annotated datasets. An industry of data and annotation services has emerged to serve the development of such data-intensive automotive software components. Wide-spread difficulties to specify data and annotation needs challenge collaborations between OEMs (Original Equipment Manufacturers) and their suppliers of software components, data, and annotations. This paper investigates the reasons for these difficulties for practitioners in the Swedish automotive industry to arrive at clear specifications for data and annotations. The results from an interview study show that a lack of effective metrics for data quality aspects, ambiguities in the way of working, unclear definitions of annotation quality, and deficits in the business ecosystems are causes for the difficulty in deriving the specifications. We provide a list of recommendations that can mitigate challenges when deriving specifications and we propose future research opportunities to overcome these challenges. Our work contributes towards the on-going research on accountability of machine learning as applied to complex software systems, especially for high-stake applications such as automated driving.