Best Practice Using a Dermatology Search Engine

by | Jun 3, 2018 | Artificial Intelligence (AI), Blog, News

Skin Image Search Artificial Intelligence AI Dermatology how it works

On this image you can see where the Convoluted Neural Network hones in on the skin lesion of importance

Skin Image Search™ Best Practice

Skin Image Search™ is an artificial intelligence (AI) algorithm that has evolved from training a Convolutional Neural Network (CNN) with hundreds of thousands of amateur smartphone pictures of skin diseases. In order for it to work well, it needs similar pictures that it has been trained on to give a good answer back.


Take a good, focused and true picture of the area of concern

Skin Image Search™ is very good at identifying what is in a picture and sometimes it can be too good in regards to identifying things on your skin. When you have a skin concern, the body can be covered by other skin lesions that are not of relevance to your concern, but the computer does not understand that. For example, the body is covered by hair, you might have previous scarring, you may have pimples or anatomically irrelevant structures, such as your belly button.

This is a problem if you would like to get a good answer back. The Skin Image Search™ algorithm can mistake body parts for another area of your body. For example, different angles of the nose, a single finger or a toe can be confused as a penis; to the computer they look similar. The Skin Image Search™ has been trained on pictures and compares your picture to that of the dataset, and tries to find something similar.

Try our FREE skin image search and get the right answer now

The search will progressively get better over time as the CNN is trained on more skin pictures from all over the body and a new improved algorithm develops.

Another problem is that if you take a picture of normal skin or something that is not related to skin, like a car; you will still get an answer in regards to a skin concern. Skin Image Search™ has at this stage not been trained on non-skin diseases to be able to return a message that it is not a skin problem, nevertheless this will be possible in the future.

Dos and Don’ts when using Skin Image Search™

  • Do use a smartphone with a good camera
  • The picture size pixels Mb does not matter
  • Do use good natural lighting
  • Do clearly capture the area of concern
  • Do take focused pictures
  • Don’t have irrelevant distractions in the picture, like a watch, jewelry, pointing your finger or draw on the picture

Try our FREE skin image search and get the right answer now

Case study

The example below describes a user that was wondering what the red colored skin lesions on his trunk were. One of our First Derm dermatologists answered it as Pityriasis Versicolor – a fungal infection that can be easily treated with a prescription free schampo from a pharmacy.

Pityriasis versicolor Skin Image Search AI_1                 Pityriasis versicolor Skin Image Search AI_2

The user was concerned about the red patches on his torso.The user sent in two pictures. If you look at the pictures there is hair, a belly button, red patches and also an uneven dark mole. The lighting could possibly have been better. We uploaded the images to Skin Image Search™.

Skin Image Search™ Results

Pityriasis versicolor Skin Image Search AI_5

The top result was acne, which has nothing to do with the case. The computer was distracted by the enormous amount of features in the image. An observation is that it managed to hone in on the darker mole. The answer reflects that it was not focused on one skin disease, but several. If the user clicks on the respective skin disease links, he can read more on the skin diseases and will hopefully help guide him in the right direction.

Try our FREE skin image search and get the right answer now

Results with cropped, focused pictures of the area of concern

Cropped Pityriasis versicolor Skin Image Search AI

A new search is done with the correctly cropped pictures focusing on the skin concern and a new list of answers comes up of what it could possibly be.

Pityriasis versicolor Skin Image Search AI_7

The cropped pictures focuses on the red patches, which gives the top answer as pityriasis versicolor, which was also confirmed by a dermatologist.

Try our FREE skin image search and get the right answer now


Results with the cropped out dark mole

Cropped Seborrheic Keratosis Skin Image Search AI

A new search focused on the dark mole, nevertheless the red patches are still in the picture.

Seborrheic Keratosis Skin Image Search AI_2

The cropped image focuses on the mole and it reveals that the user possibly has a seborrheic keratosis. Ranked as number two is malignant melanoma as a differential answer. This mole is something that should get checked out by an in person visit to a dermatologist, with the use of a dermatoscope which can determine a correct diagnosis and if it should be removed, biopsied or left under observation.

Try our FREE skin image search and get the right answer now

Skin Image Search™ accuracy

Version 1.0 of our web interface is trained on 33 skin disease classes (inflammatory, moles and STDs). The accuracy is 40% on one skin disease and 80% for top 5 skin diseases. We are continuously experimenting to make it better. Within 9 months the goal is to be 80% accurate on one skin disease and 100% on top 3 skin diseases.

We have an open API that can integrate into any internet platform and apps. Currently health chat bots and telemedicine primary care clinic use the API. If interested you can sign up here


Try our FREE skin image search and get the right answer now

Over 15,000+ Readers

Get fresh content from First Derm

Ask a Dermatologist

Anonymous, fast and secure!

1 (415) 234-4124
Share This