Google dermatology AI detects 26 Skin Conditions as Accurately as Dermatologists
Image Credit: Google AI Accra – AI in Africa, for the world
Skin Care AI Technology
We pride ourselves in our Artificial Intelligence (AI) technology at First Derm and we have exciting news to share from Google. This week Google has opened up discussions on the difficulties faced by sufferers of skin conditions and the challenges dermatologists are forced to endure. This is compounded by the global shortage of dermatologists and the strain they are therefore put under. Globally, Skin conditions are among the most common kind of ailment, just behind colds, fatigue, and headaches. In fact, around the world it’s estimated that 25% of all treatments provided to patients are for skin conditions and that up to 37% of patients seen in the clinic have at least one skin complaint.
Due to the shortage in Dermatologists, as we all know too well, General Practitioners (GPs) are asked to pick up the slack. This can often result in misdiagnoses and GPs will typically see a success rate as low as 24% in diagnosing skin conditions unlike up to 96% accuracy for Dermatologists. So what does this all mean to Google?
They began to investigate an AI system capable of spotting the most common dermatological disorders seen in primary care. Not unlike our very own AI technology here at First Derm. In a paper (“A Deep Learning System for Differential Diagnosis of Skin Diseases“) and accompanying blog post, they report that it achieves accuracy across 26 skin conditions when presented with images and metadata about a patient case, and they claim that it’s on par with U.S. board-certified dermatologists. It’s great to see Google stepping up their pursuit in healthcare technology and to have achieved accuracy on 26 skin conditions is some going. Some way to go before they reach our 33 skin conditions… Google states:
“We developed a Deep Learning System (DLS) to address the most common skin conditions seen in primary care,” wrote Google software engineer Yuan Liu and Google Health technical program manager Dr. Peggy Bui. “This study highlights the potential of the DLS to augment the ability of general practitioners who did not have additional specialty training to accurately diagnose skin conditions.”
Above: A schematic illustrating the AI system’s architecture.
Image Credit: Google
Using A Dermatologist Ranking System
Liu and Bui then went on to explain that dermatologists don’t give just one diagnosis for any skin condition — instead, they generate a ranked list of possible diagnoses (a differential diagnoses) to be systematically narrowed by subsequent lab tests, imaging, procedures, and consultations. This is exactly how our technology works here at First Derm and it appears Google have also followed suit. They process inputs that include one or more clinical images of the skin abnormality and up to 45 types of metadata (e.g., self-reported components of the medical history, such as age, sex, and symptoms).
According to the research team at Google, they say they evaluated the model with 17,777 de-identified cases from 17 primary care clinics across two states. They bifurcated the corpus and used the portion of records dated between 2010 and 2017 to train the AI system, reserving the portion from 2017 to 2018 for evaluation. During training, the model leveraged over 50,000 differential diagnoses provided by over 40 dermatologists.
In a test of the system’s diagnostic accuracy, the researchers compiled diagnoses from three U.S. board-certified dermatologists. Just over 3,750 cases were aggregated to derive the ground truth labels, and the AI system’s ranked list of skin conditions achieved 71% and 93% top-1 and top-3 accuracies, respectively. Furthermore, when the system was compared against three categories of clinicians (dermatologists, primary care physicians, and nurse practitioners) on a subset of the validation data set, the team reports that its top three predictions demonstrated a top-3 diagnostic accuracy of 90%, or comparable to dermatologists (75%) and “substantially higher” than primary care physicians (60%) and nurse practitioners (55%).
For us at First Derm it’s refreshing to see this data taken seriously, Dermatologists and Doctors can frequently misdiagnose skin conditions, which is where our AI steps in. It appears that Google is achieving similar accuracy results to us but with fewer images leading to a more limited range of skin diagnoses.
Above: The AI system’s accuracy, trained on different data sets.
Image Credit: Google
Lastly, in order to evaluate potential bias toward skin type, the team tested the AI system’s performance based on the Fitzpatrick skin type, a scale that ranges from Type I (“pale white, always burns, never tans”) to Type VI (“darkest brown, never burns”). Focusing on skin types that represent at least 5% of the data, they found that the model’s performance was similar, with a top-1 accuracy ranging from 69% to 72%, and a top-3 accuracy from 91% to 94%.
They note that their training corpus was only taken from a one teledermatology service; that some Fitzpatrick skin types were too rare in their data set to allow meaningful training or analysis; and that their data set didn’t accurately detect some skin conditions, such as melanoma, due to a lack of available data samples. We did a study with a group from Uganda in Africa that tested our AI on their patients, you can read the biased results here
Google Dermatology AI Limitations
Liu and Bui mentioned, “We believe these limitations can be addressed by including more cases of biopsy-proven skin cancers in the training and validation sets,” – we believe this to be absolutely correct since it’s an area we have covered in great depth here at First Derm. Due to our wealth of varied and detailed images within our algorithm we are able to successfully test for Malignant Melanoma with the highest levels of accuracy.
Further to our work on Melanoma we have also focused efforts on improving the testing capabilities within visual STD’s. After years of data collection and algorithm advancements we are now able to test for all visual STDs with over 80% accuracy.
Liu and Bui go on to conclude their study with a fair assessment of our growing market.
“The success of deep learning to inform the differential diagnosis of skin disease is highly encouraging of such a tool’s potential to assist clinicians. For example, such a DLS could help triage cases to guide prioritization for clinical care or help non-dermatologists initiate dermatologic care more accurately and could potentially improve access [to care].”
At First Derm we aim to significantly improve the tools available for our dermatologists and GPs both locally and globally. The vast number of Skin condition cases per year leaves a clear necessity for technological support in the long-term future. That is where we believe First Derm can provide the answer.
We have released an API for our AI in dermatology that you can read more about, how to implement with just 100 lines of code.
Related article, read how LEO Innovation announced they have beaten Google on artificial intelligence and dermatology.
Ask a Dermatologist
Anonymous, fast and secure!
Specialist doctor from the University Hospital in Gothenburg, alumnus UC Berkeley. My doctoral dissertation is about Digital Health and I have published 5 scientific articles in teledermatology and artificial intelligence.