Self-Driving Cars May be More Likely to Hit Darker-Skinned People, Study Finds

Self-driving cars used to only be a thing found in movies, but now they’re making their way onto the streets. That seems cool, but there’s a growing list of concerns about them.

Recently, a study from the Georgia Institute of Technology found a person with darker skin is more likely to be hit by a self-driving car than white people.

For the study, researchers looked at eight AI models that are used in state-of-the-art object detection systems. As noted by Futurism, similar systems are what allows self-driving cars to see roads, pedestrians, and other objects.

They tested the AI models by pulling images of pedestrians from a large dataset. The images were divided using the Fitzpatrick scale, a classification system measuring melanin in skin.

Researchers saw “uniformly poorer performance of these systems when detecting pedestrians with Fitzpatrick skin types between 4 and 6.” When it came to dark-skinned pedestrians, the models’ accuracy decreased by five percentage points. That didn’t change even when researchers accounted for variables like time of day or obstructed view of the pedestrians.

The study does have its own flaws, as outlined by Vox, including: lack of peer review; it didn’t have access to datasets used to train self-driving cars; and it didn’t test an AI system actually used by self-driving cars.

Still, the study is valuable and its results aren’t actually surprising. Previous research has found other AI programs can’t detect dark skin, either. For example, a study reported in the New York Times found facial-recognition systems from IBM, Microsoft, and China’s Megvii were more likely to misidentify the gender of darker-skinned women.

In addition, the ACLU found Amazon’s Rekognition system falsely matched 28 members of Congress with mugshots. The system disproportionately issued false matches to people of color, including six members of the Congressional Black Caucus.

Jamie Morgenstern, one of the study’s authors, told Vox, “The main takeaway from our work is that vision systems that share common structures to the ones we tested should be looked at more closely.”