ScholarWorks@경상국립대학교

초록

Text summarization and content tagging are pivotal Natural Language Processing (NLP) tasks, enhancing information accessibility, organization, decision-making and optimizing utilization of data for diverse applications. This paper addresses the problem of automating these tasks by evaluating and comparing general-purpose and specialized models. For text summarization, a general-purpose large language model (LLM) is used and compared against specialized models such as Bidirectional and Auto-Regressive Transformer (BART) and Pegasus, focusing on accuracy, coherence, and relevance. BART achieved the highest ROUGE-1 score of 44.2553, highlighting its strong performance in abstractive summarization. For content tagging, the Bidirectional Encoder Representations from Transformers (BERT) model is evaluated on a classification dataset and benchmarked against other state-of-the-art models, including Robustly Optimized BERT approach (RoBERTa), DistilBERT, and XLNet, to assess accuracy, speed, and applicability. BERT outperformed other models for content tagging, achieving an accuracy of 0.9315, precision of 0.9393, and recall of 0.9263. The major contributions include identifying trade-offs between general-purpose and specialized models, providing recommendations for real-world applications such as news aggregation and content management systems.

키워드