Video walkthrough

A protocol for annotation of total body photography for machine learning to analyze skin phenotype and lesion classification

Overview 3:59 CC AI

paperi.ai
0:00 / 0:00

Clare Primiero, Brigid Betz‐Stablein, Nathan Ascott, Brian D’Alessandro, Séraphin Gaborit, Paul Fricker, Abigail Goldsteen, Sandra González-Villà, Katie L. Lee, Sana Nazari, Hang Thanh Nguyen, Valsamis Ntouskos, Frederik Pahde, Balázs Pataki, Josep Quintana, Susana Puig, Gisele Gargantini Rezze, Rafael García, H. Peter Soyer, Josep Malvehy

A skin-scanning system can be good at judging one spot, yet still miss the person around it. This paper asks whether the whole body, medical history, and change over time can make that judgment safer.

Abstract

Introduction: Artificial Intelligence (AI) has proven effective in classifying skin cancers using dermoscopy images. In experimental settings, algorithms have outperformed expert dermatologists in classifying melanoma and keratinocyte cancers. However, clinical application is limited when algorithms are presented with ‘untrained’ or out-of-distribution lesion categories, often misclassifying benign lesions as malignant, or misclassifying malignant lesions as benign. Another limitation often raised is the lack of clinical context (e.g., medical history) used as input for the AI decision process. The increasing use of Total Body Photography (TBP) in clinical examinations presents new opportunities for AI to perform holistic analysis of the whole patient, rather than a single lesion. Currently there is a lack of existing literature or standards for image annotation of TBP, or on preserving patient privacy during the machine learning process. Methods: This protocol describes the methods for the acquisition of patient data, including TBP, medical history, and genetic risk factors, to create a comprehensive dataset for machine learning. 500 patients of various risk profiles will be recruited from two clinical sites (Australia and Spain), to undergo temporal total body imaging, complete surveys on sun behaviors and medical history, and provide a DNA sample. This patient-level metadata

Transcript

A skin-scanning system can be good at judging one spot, yet still miss the person around it. This paper asks whether the whole body, medical history, and change over time can make that judgment safer. Computer programs can classify skin cancers from close-up skin images, and in experimental settings they have outperformed expert dermatologists at classifying some skin cancers.

But clinical use is limited when a program meets unfamiliar kinds of lesions. It can mistake a harmless lesion for cancer, or a cancerous lesion for something harmless. The wider problem is context: these tools often lack medical history, even though whole-body photography could support analysis of the whole patient rather than a single lesion.

So the project builds a database of skin images enriched with information about lesion categories, skin appearance, and details about each patient. Like teaching a detective with more than one photograph, the dataset combines skin images with information about skin phenotype and each patient’s background.

That added context supports algorithms designed to assess risk for an individual lesion and for the patient as a whole. That information is intended to support algorithms assessing risk for an individual lesion and for the patient as a whole. The master database is then divided into separate training, test, and validation datasets for developing and checking the resulting programs.

The collected material is meant to train deep machine-learning models for several linked goals. Here, the training dataset is used to design and develop deep machine learning models for the study’s objectives. The planned programs will add annotations to previously unseen whole-body images and identify changes in a patient’s skin images over time.

They will locate lesions, outline them, match lesions across examinations, identify unusual lesions, classify them, and detect changes within and between examinations. Those results are used to extract visible patterns of skin and lesions for assessing lesion risk.

The risk assessment uses information about lesion size, color, diagnosis, and skin tone, along with patient risk scores. It produces a final risk score for each identified lesion, ranging from zero to one, with zero indicating healthy skin and one indicating high risk of skin cancer.

The score is based on the severity of detected skin-disease characteristics and incorporates established knowledge about specific skin diseases and conditions. At the end of the study, the planned dataset will contain anonymized whole-body images labelled with patient-level information, including skin appearance and lesion diagnostic categories.

Patient-level information will allow machine-learning protocols to evaluate an individual’s risk for melanoma, while the planned tools track lesions, match earlier images, detect change, and assess unusual lesions. Together, those tools contribute to lesion risk assessment, while a second whole-patient risk program uses patient information and skin appearance.

The protocol brings together repeated whole-body photography, lesion classification, skin appearance, and patient-level information in one machine-learning dataset. Its intended outcome is a whole-person approach to computer-assisted tools for early melanoma detection, including clinical context that a dermatologist would normally consider.

Combined with a whole-body scanning platform, these tools are intended to improve patient risk assessment and surveillance for early signs of skin cancer. The central idea is to give skin-checking tools the same wider context a dermatologist uses: the person, their skin, and how lesions change.

That could make suspicious changes easier to find and follow.

A derivative work by Paperi · AI-generated script, voice and captions · pages and figures unaltered

Made with Paperi.

Drop in a research PDF — get a narrated video walkthrough like this one, with highlights that follow the narration. Free to start.

Try it with your paper →

More in Medicine

Modelling the historical distribution of schistosomiasis-transmitting snails in South Africa using ecological niche models 3:59

Modelling the historical distribution of schistosomiasis-transmitting snails in South Africa using ecological niche models

A disease carried by freshwater snails can move when the weather moves. By tracing where the snails could live, researchers mapped places in South Africa where infection may already be hiding, and where it could spread next.

Wearable derived cardiovascular responses to stressors in free-living conditions 4:25

Wearable derived cardiovascular responses to stressors in free-living conditions

Stress does not necessarily end when the stressful moment ends. A wearable may be able to see the body remain changed long after a hard run or a tense period of work.

A hybrid register and questionnaire study of Covid-19 and post-acute sick leave in Denmark 3:46

A hybrid register and questionnaire study of Covid-19 and post-acute sick leave in Denmark

Covid can end as an emergency and still leave a mark on ordinary working life. In Denmark, people who had the infection were more likely to spend a substantial stretch of time away from work afterward.

All 50 papers in Medicine →