HyLME: Language Model Embeddings with Knowledge Distillation for Robust Predictive Maintenance under Missing Sensor Data
Citations

SCOPUS

0

초록

In the 4th Industrial Revolution, predictive maintenance is a key strategic element that predicts equipment failure or status in real-time by collecting relevant data from sensors. However, missing sensor data occurs frequently, leading to enormous opportunity costs. Therefore, there is a growing need for robust models that can maintain high accuracy and stable performance even when missing data occurs. Against this backdrop, language models have achieved remarkable success in natural language processing tasks through the innovative architecture of Transformers, and their context-based learning effect has been widely proven in other domains such as tabular data and time-series analysis. In this paper, we propose a Hybrid architecture with a machine learning teacher and a Language Model Embedding-based student (HyLME) to enable accurate and robust predictive maintenance even in the presence of missing data. HyLME is a hybrid learning approach that fuses text embeddings from language models and learns knowledge from tree-based machine learning models. Additionally, we implemented a masking scenario based on feature importance to simulate missing data conditions. In the results of comparative experiments, HyLME achieved an average accuracy of 0.83207 across all scenarios. This performance is 15.99% higher than the average accuracy of the comparison models, which indicates that the proposed architecture is effective in performing accurate and robust predictions in sensor missing data situations. © 2025 IEEE.

키워드

hybrid deep learning and machine learningknowledge distillationlanguage model embeddingsmissing data processingpredictive maintenance
제목
HyLME: Language Model Embeddings with Knowledge Distillation for Robust Predictive Maintenance under Missing Sensor Data
저자
Kim, Ju-YoungPark, Ji-HongKim, Gun-Woo
DOI
10.1109/ICTC66702.2025.11387994
발행일
2026-02
유형
Conference paper
저널명
International Conference on ICT Convergence
페이지
691 ~ 696