Automatic Identification of Regions of Interest in Dermoscopy Images Using Vision Transformers and Weakly Supervised Learning

24/11/2023

Skin cancer is a growing public health concern. Early detection of the lesion plays a critical role in ensuring successful treatment of the cancer. Dermatologists traditionally use criteria like the 7-point checklist, which focuses on specific dermoscopic characteristics without considering their spatial distribution in the lesion. Multiple Instance Learning (MIL) is a weakly supervised learning technique that serves as an approximation to this criterion in the field of deep learning. In contrast to these methods, Vision Transformers (ViTs) have recently shown remarkable promise, while at the same time using spatially aware information from all the patches in the image. This contrast motivates us to address two questions in dermoscopy image analysis: (1) the understanding of whether all patches are relevant for skin cancer diagnosis, and (2) the influence of the spatial arrangement of the patches on diagnostic accuracy. To address these questions, we introduce a two-branch framework that combines a ViT-based architecture with a MIL model. We tackle both binary classification (melanoma vs. nevus) and multi-class classification (with eight skin disease types). Our work presents a novel two-stage MIL formulation oriented towards binary classification, and we extend it to a three-stage approach for multi-class classification. Our results consistently demonstrate the competitive performance of these formulations in both binary and multi-class contexts. Our findings reveal that only certain patches are critical for correct classification, and that adding spatial information slightly improves classification accuracy.

Autores da comunidade :

• 

  D

  Diogo José Pereira Araújo

  ist193906

  Afiliações e fontes

  IST > LARSYS > ISR

  Instituto de Sistemas e Robótica

  De: cienciavitae

  ID: 4045012

Orientadores desta instituição:

• 

  C

  Carlos Jorge Andrade Mariz Santiago

  ist158445

• 

  A

  Ana Catarina Fidalgo Barata

  ist158472

- Engenharia Eletrotécnica, Eletrónica e Informática

- Inglês

Instituto Superior Técnico