교육평가에서 ROC 분석의 활용 가능성 탐색: 분할 점수 설정과 분류 정확도를 중심으로

Applications of ROC Analysis in Educational Assessment: Implications for Cut Score Setting and Classification Accuracy

초록

This study explored the applicability of ROC analysis in educational assessment and examined the characteristics of ROC-based cut score setting for identifying students at risk of academic underachievement. A simulation study was conducted under multiple conditions involving prevalence rates, effect sizes, sample sizes, and score distribution shapes. ROC-based cut scores derived from the Youden Index and the Euclidean distance method were compared with expert-established cut scores. Classification performance was evaluated using the AUC, sensitivity, specificity, positive predictive value(PPV), and negative predictive value(NPV). The results showed that AUC remained relatively stable across different prevalence conditions, whereas PPV decreased substantially as the prevalence of at-risk students declined. In addition, ROC-based cut scores diverged from expert-established cut scores under skewed distributions and small sample sizes, increasing the likelihood of false-positive and false-negative classifications. These findings indicate that the performance of ROC-based cut scores is influenced by data characteristics and assessment contexts. Overall, ROC analysis may serve as an empirical validation tool that complements expert-based standard-setting procedures by providing evidence on classification accuracy and the practical validity of cut scores in educational decision-making.

키워드

준거 설정; ROC 분석; 분할 점수; 모의 실험; Standard Setting; ROC Analysis; Cut Score; Simulation
제목
교육평가에서 ROC 분석의 활용 가능성 탐색: 분할 점수 설정과 분류 정확도를 중심으로
제목 (타언어)
Applications of ROC Analysis in Educational Assessment: Implications for Cut Score Setting and Classification Accuracy
저자
이문수
발행일
2026-09
유형
Y
저널명
문화기술의 융합
권
12
호
5
페이지
161 ~ 170