멀티모달 데이터를 활용한 멀티 헤드 크로스 어텐션 기반 피부 병변 분류 모델

초록

In this paper, we propose a skin lesion classification model utilizing multimodal data. For accurate diagnosis of skin lesions, we combined lesion images, lesion mask images, and metadata, and applied a multi-head cross-attention mechanism. We conducted model training and classification experiments using 26,526 data points and 5 skin lesion classes compiled from the International Skin Imaging Collaboration dataset. Our proposed model showed an improvement of 2.5 percentage points in accuracy and 2.7 percentage points in F1-Score compared to single models. Additionally, analysis of the Receiver Operating Characteristic curve indicated that the proposed model achieved an average Area Under the Curve value of over 0.98 for each class, confirming its effectiveness in skin lesion classification. This suggests that more accurate skin lesion classification is possible by combining multimodal data.

키워드