Can ChatGPT Be Trusted for Urologic Patient Education? A Comparative Study of Stress Urinary Incontinence-Related Information From Versions 3.5 and 4

초록

Introduction and Hypothesis The application of artificial intelligence tools such as ChatGPT in patient education is expanding rapidly. Female stress urinary incontinence (SUI) is a common yet often overlooked urological condition. However, the accuracy, comprehensiveness, and evidence-based reliability of ChatGPT's responses to common SUI-related questions remain unclear. Methods On the basis of the AUA/SUFU clinical practice guidelines, 22 frequently asked questions regarding female SUI were developed and input into ChatGPT-3.5 and ChatGPT-4, respectively. Two senior urologists independently evaluated the responses using a 5-point Likert scale for accuracy, comprehensiveness, and relevance. Additionally, the word count of each response was recorded, and the validity of any cited references was verified. Results A total of 44 AI-generated responses were analyzed for the 22 SUI-related questions. Both ChatGPT-3.5 and ChatGPT-4 provided high-quality medical information, with accuracy scores of 4.09 and 4.45, respectively (p = 0.097). ChatGPT-4 offered significantly more concise responses (200.55 +/- 48.21 words) compared to ChatGPT-3.5 (515.68 +/- 198.13 words; p < 0.001). Furthermore, ChatGPT-4 demonstrated a significantly higher proportion of valid citations (72.06% vs. 24.27%, p < 0.001). Conclusions ChatGPT-4 demonstrated strong performance in delivering accurate, concise, and evidence-supported information on female SUI. Future research should expand the scope of evaluation, incorporate patient perspectives, and validate the practical utility and safety of AI tools in real-world clinical settings.

키워드