Malware classification with disentangled representation learning of evolutionary triplet network

Abstract

Malware is a significant threat to the security of computer systems and networks worldwide, and its sophistication and diversity continue to increase over time. One of the key challenges in malware detec-tion and classification is the high variability and similarity of the malicious code. This paper proposes a novel method for malware classification with disentangled representation from an evolutionary triplet network. We aim to learn a representation of malware samples that captures the underlying factors of variation, making it easier to distinguish between different malware types. The genetic algorithm-based optimization enables us to find the optimal distance representation of malware, which helps to minimize the intra-class distance and maximize the inter-class distance in the disentangled space. By evolutionary optimization of the triplet network, our model is able to better capture the subtle differ-ences in the structural characteristics of malware, which led to significant improvements of classification accuracy and recall in three benchmark datasets. Furthermore, this method demonstrates significant improvement on t-SNE visualization, indicating that the learned features are more discriminative and better capture the underlying structure of the malware.& COPY; 2023 Elsevier B.V. All rights reserved.