For years, tech companies have relied on something called the Fitzpatrick scale to classify skin tones for their computer vision algorithms. Originally designed for dermatologists in the 1970s, the system only includes six skin tones, possibly contributing to AI’s well-documented failures in identifying people of color. Now, Google is starting to include a 10-skin tone standard in its products, called the Monk Skin Tone (MST) scale, from Google Search Images to Google Photos and more. The development has the potential to reduce bias in datasets used to train AI in everything from healthcare to content moderation.
Google first signaled plans to go beyond the Fitzpatrick scale last year. Internally, the project traces back to an effort by four black women at Google in the summer of 2020 to make AI “work better for people of color,” according to one Twitter thread from Xango Eyeé, a responsible AI product manager at the company. At today’s Google I/O conference, the company outlined how big the impact the new system could have on its many products. Google will also make the MST open source, meaning it could replace Fitzpatrick as the industry standard for evaluating the fairness of cameras and computer vision systems.
“Remember that images of people’s faces are used everywhere we need to test the algorithm for fairness,” Eyeé says.
The Monk Skin Tone scale is named after Ellis Monk, a sociologist at Harvard University who spent decades researching the impact of colorism on the lives of black people in the United States. Monk created the scale in 2019 and worked with Google engineers and researchers to incorporate it into the company’s product development.
“The reality is that life chances, opportunities, all of these things are strongly linked to your phenotypic makeup,” Monk said in prepared comments in a video shown at I/O. “We can get rid of these biases in our technology very early on and make sure the technology we have works equally well for all skin tones. I think this is a huge step forward.”
An initial analysis last year by research scientists at Monk and Google found that participants felt better represented by the MST than by the Fitzpatrick scale. In an FAQ published Wednesday, Google says having more than 10 skin tones can add complexity without adding value, unlike industries like makeup, where companies like Rihanna’s Fenty Beauty offer more than 40 shades. Google continues to work on validating the Monk Skin Tone scale in places like Brazil, India, Mexico and Nigeria, according to a source familiar with the matter. Further details are expected soon in a scientific research paper.
The company will now expand the use of the MST. Google Images offers an option to sort makeup-related search results by skin tone based on the scale, and filters for those with more melanin are coming to Google Photos later this month. If Google were to use the 10-skin-tone scale across its product lines, it could have implications for fairly evaluating algorithms used in Google’s search results, Pixel smartphones, YouTube rating algorithms, Waymo’s self-driving cars and more.
