Comparative Analysis of the Readability of Artificial Intelligence-Supported Educational Materials for Patients with Tubo-ovaryan Apse: ChatGPT versus Gemini
Background: Tubo-ovarian abscess (TOA) is a serious gynecological emergency requiring timely diagnosis, appropriate treatment, and effective patient education. Large language models (LLMs), such as ChatGPT and Gemini, are increasingly used to provide medical information; however, the readability of their educational content remains insufficiently investigated.
Objective: This study aimed to compare the readability of patient educational materials on tubo-ovarian abscess generated by ChatGPT and Gemini using multiple validated readability indices.
Methods: In this cross-sectional comparative study, 50 frequently asked questions regarding tubo-ovarian abscess were submitted independently to ChatGPT and Gemini, resulting in 100 AI-generated educational texts. Responses were analyzed using nine validated readability indices, including the Automated Readability Index (ARI), Flesch Reading Ease Score, Flesch–Kincaid Grade Level, Gunning Fog Index, Coleman–Liau Index, SMOG Index, Linsear Write Formula, Dale–Chall Readability Score, and Spache Readability Formula. Continuous variables were expressed as mean ± standard deviation, and readability scores were compared using appropriate statistical tests. A p-value <0.05 was considered statistically significant.
Results: Most readability indices demonstrated comparable performance between ChatGPT and Gemini. No statistically significant differences were observed in ARI (p=0.664), Flesch Reading Ease (p=0.364), Gunning Fog Index (p=0.468), SMOG Index (p=0.464), Linsear Write Formula (p=0.112), Dale–Chall Readability Score (p=0.148), or Spache Readability Formula (p=0.684). However, Gemini generated significantly more readable educational materials according to the Flesch–Kincaid Grade Level (6.46±2.64 vs. 9.66±4.64; p<0.001) and Coleman–Liau Index (11.64±3.64 vs. 16.24±2.64; p=0.012). Despite these differences, both AI models produced texts requiring reading abilities above the sixth-grade level recommended for patient education.
Conclusions: Both ChatGPT and Gemini generated informative educational materials for patients with tubo-ovarian abscess; however, their readability generally exceeded recommended health literacy standards. Gemini demonstrated superior readability in selected grade-level indices, suggesting that it may provide more accessible patient education than ChatGPT. Nevertheless, further refinement of AI-generated medical content is necessary to improve accessibility for individuals with limited health literacy while maintaining clinical accuracy.
Keywords: Tubo-ovarian abscess; Artificial intelligence; ChatGPT; Gemini; Readability; Health literacy; Patient education; Large language models.
